Stripe is building a Ruby typechecker - <AUTHOR> https://medium.com/byteconf/stripe-is-building-a-ruby-typechecker-d6cd7cee6abf <comment><USER3> "Why don't they just use (insert static-typed system here) instead?" someone will inevitably ask.

"Because converting a large, complex set of living codebases from one language to another is a non-trivial task that requires boiling any number of oceans. This approach doesn't have that problem." comes the wiser, if less interesting, response.

<comment><USER4> I'm one of those people. I'm not sure I'd keep loving Ruby as much as I do now if the majority of Ruby developers end up using the type checker and I'll also have to. I like Ruby in part because it's strongly typed but I don't have to lose time writing types. That was very welcome coming from C and Java in 2006. I prefer to lose time when I pass a type that I shouldn't have passed. It doesn't happen often (not every year) vs at every single method definition.

To understand if this project could have a future outside Stripe, how popular is Python 3.6's optional static type declaration?

<comment><USER5> > I like Ruby in part because it's strongly typed but I don't have to lose > time writing types.

That's why _type inference_ exists. Static typing doesn't imply annotating everything manually with types. Granted, _most_ static type systems require (or at least encourage) this but it's not necessary everywhere (or indeed for most usage). Annotating public interfaces with types is, I'd argue strongly, already best practice everywhere (in languages that don't support it in the language, it's done in documentation).

> To understand if this project could have a future outside Stripe, how > popular is Python 3.6's optional static type declaration?

Or rather: How popular are TypeScript or Flow? _Very_.

<comment><USER6> Ruby has the advantage of 'duck typing', due to its strong OOP foundations. Not quite sure what Python has that reduces the demand for static type declaration.

I think part of the popularity of TypeScript could be down to a large proportion of javascript coders that have a background/preference for static typing. It could also be that certain classes of bugs are more prone to happen in JavaScript or it's harder to debug when bugs do happen?

<comment><USER7> Ruby has been my first professional language and I've sticked with it for a decade, but I never really understood duck typing before switching to Go, with its interfaces (a type implements an interface if it implements its methods, so we can use interface as function parameter type).

I don't get why we talk of duck typing about ruby : if I pass as parameter an object that does not quack like a duck, nothing will prevent me to do it and it will generate an exception when quacking. Did I miss something obvious?

<comment><USER8> The obvious part is that Ruby does not have a distinction between "compile time" and "runtime". As such, if you want to catch those cases, you need tests.

Duck-typing can be boiled down to: The type of a parameter to a function is defined by the methods that will be called on it, not by its class.

E.g. if you have:

        def foo source
          dosomething(source.shift)
        end

Then absent additional restrictions inferred from requirements of dosomething(), the class of "source" is irrelevant. The relevant type information is whether or not "source" responds to "shift".

A proper Ruby-ish type checker will need to be able to handle that, or it'll push people to write non-idiomatic Ruby.

<comment><USER9> > A proper Ruby-ish type checker will need to be able to handle that, or it'll > push people to write non-idiomatic Ruby.

It probably won't be idiomatic (maybe just add interfaces?) but I think duck- typing is overrated anyway. Just because an object responds to a method doesn't mean it's going to do anything like what you expect. To take a contrived example, Array.Shift and Keyboard.Shift are probably unrelated methods.

<comment><USER8> You don't have that guarantee with interfaces either - you're trusting that the developer isn't lying to you. My experience is that I've more often run into developers that have artificially barred me from doing what I wanted by checking for a given class, than that I've accidentally passed in something that satisfies a given type but does something different.

<comment><USER5> > You don't have that guarantee with interfaces either

True, but you can give the interfaces/contracts proper names (or namespaces). To use the example in the parent comment, `Array` and `Keyboard` wouldn't implement the same contract even if both have a `Shift` method.

</comment></comment></comment></comment></comment></comment></comment></comment></comment><comment><AUTHOR> Not yet open-sourced, but this is definitely planned.

You can try it out there though: [https://sorbet.run](https://sorbet.run)

By the way, a number of RubyKaigi 2018 talks were dealing with type checking (and the other were about improving performance & memory usage).

<comment><USER10> I had the "pleasure" of creating and integrating a parser for a custom DSL into ACE. I've created the parser in ~3 days; it took another 3 to make it work with ACE.

If it was up to me, I'd chose monaco editor (as the guys from mozilla did it with their webassembly editor)

<comment><USER11> (author of the demo site here)

I just was looking for a quick editor for the demo, and ace seemed to work well for this. Thanks for the pointer to manaco, I'll look at that for next time.

Using an editor in the browser won't be the final product. We're planning on integrating into your editor of choice instead.

</comment></comment></comment><comment><USER11> Thank you so much for all the interest! We're flattered and excited to see all the discussion about the project.

Try it out: [https://sorbet.run](https://sorbet.run)

If you would like to get in touch with us about anything, please email us at firstname.lastname@example.com.

In the presentation at RubyKaigi (which will be available online soon) we explicitly mentioned we'd like to chat with folks trying to scale Ruby into the millions of lines of code, or folks also working on similar typechecking projects. Of course feel free to email even if you aren't in those groups, but I wanted to get your attention if you were.

<comment><USER8> One thing that springs to mind: Does it handle duck-typing? In other words, can I specify "any object that responds to methods x and y" as the expected type of a variable?

Because if treats classes as equivalent to types, then to me it encourages non-idiomatic Ruby.

<comment><USER12> We only have the published examples to go on, but unfortunately they all seem to assume that type means class.

If that's borne out I could never use this library. It'd be the antithesis of duck typing and object messaging.

</comment></comment></comment><comment><USER13> So you know when people ask "why does X startup have X,000 developers? what do they do all day for a single app[0]. Here is your answer:

"Technical details: \- 9 month of work by 3 people; \- real thing, runs over all code of #Stripe"

[https://twitter.com/darkdimius/status/1002103748875902978](https://twitter.com/darkdimius/status/1002103748875902978)

Now let's look at this from a financial perspective:

So let's say average $175k salary per employee + benefits and you're looking at easily north of $600k to do this.

I'm not saying it's "right" or "wrong", but thats just what companies at this scale do because they create development cultures and subsequent financial controls that allow for indirect production software development to happen.

[0]for the record, Stripe is much more complicated than just a "single app"

</comment><comment><USER14> Any reason why existing typecheckers for Ruby didn't make the cut? There are projects like RDL [1] which provide similar functionality. I know for certain that people from Stripe were taking a look at it last year from the issues they had raised.

Disclosure: I am a grad student, recently started working on RDL.

[1]: [https://github.com/plum-umd/rdl](https://github.com/plum-umd/rdl)

<comment><USER11> Hi and thank you for working on RDL! We do love the project and spoke with Jeff Foster a few times last year.

We did use your standard library annotations for Sorbet and have many fixes to them sitting in your pull request queue: [https://github.com/plum- umd/rdl/pull/68](https://github.com/plum-umd/rdl/pull/68) [https://github.com/plum-umd/rdl/pull/72](https://github.com/plum- umd/rdl/pull/72) [https://github.com/plum- umd/rdl/pull/57](https://github.com/plum-umd/rdl/pull/57) . We stopped submitting more since these weren't being upstreamed.

Before we started our project, I evaluated rolling out RDL instead of building our own typechecker. Trust me, I would MUCH rather use an existing project than have to build our own, but sadly it just didn't scale to our millions of lines of code.

I'm more than happy to chat about the details of why we didn't use RDL if you'd like to email me at email@example.com. Thanks again for your great project, we're standing on the shoulders of giants.

<comment><USER15> You are embarking onto a very interesting, but tough project due to the extremely dynamic nature of Ruby!

Have you considered the property-based QuickCheck approach? Instead of building a type checker, you just annotate your code with properties (preconditions, postconditions and class invariants). You can generate tests to verify your code and also inject runtime checks to fail early in case of violations.

In my experience, this tends to work much better on dynamic languages and it also scales to larger codebases with heavy usage of dynamic idioms.

</comment></comment></comment><comment><USER16> This looks really nice... i wish it was just part of the language to be able to write ruby with type information e.g. instead of having a separate more verbose sig method that's called before each method signature...

``` class Foo extend T::Helpers

      sig(
          # Both positional and named parameters are referred to by name.
          # You must declare all parameters (and the return value below).
          foobar: Integer,
          widget: String
      )
      .returns(Symbol)
      def foo(foobar, widget: nil)
        (foobar.to_s + widget).to_sym
      end

end ```

I'd love it if we could write this as :

``` class Foo

      extend T::Helpers

      def foo(foobar:Integer, widget:String:nil):Symbol

        (foobar.to_s + widget).to_sym

      end

end

```

Not sure the best way to handle default values... e.g. widget:String:nil is kind of awkward... but still i'd be better this way then the additional method call 'sig'

<comment><USER11> (one of the authors here)

We'd love that too! We're chatting with the Ruby folks to see if anything like that is feasible. We experimented with lots of other syntaxes (comments, yarddoc, .rbi files, monkey-patching stdlib classes to be callable) and the `sig` syntax seems to be the easiest for folks to use inside Stripe.

After writing sigs for the past few months, I don't dislike it as much as I thought I would. You get the syntax highlighting and autocomplete of your editor, while `sig` and `T` are short enough to not feel like too much boilerplate.

We're very open to other suggestions if you have any for other syntax suggestions. Just email us at firstname.lastname@example.com.

<comment><USER17> Tomdoc?

</comment></comment></comment><comment><USER10> I know the editor is not the news, and it's used just for demo, but more info about it: currently it's an ACE editor strapped to a wasm that performs the checking. The integration looks more like a quick poc since it's not using an webworker for performing the checks nor does it use the editor's markers for errors.

</comment><comment><USER18> Is Crystal still a thing? I thought it sought to keep Ruby ease but statically for speed and to reduce bugs?

<comment><USER19> Crystal is a completely different language with a different architecture and object model that only superficially looks like Ruby. Porting code from Ruby to Crystal is somewhat easier than, say, porting the same code to Python, but is still an act of porting. You can't use any kind of automated drop-in process to rewrite Ruby code to Crystal. They're just too different.

A good type system for Ruby is very much needed. Matz himself has stated that a type system is very much on the horizon for Ruby. Can't wait.

<comment><USER20> With Ruby getting an AST, and potentially a type system in the future, I think that ruby 3 could lend itself to bring transpiled to crystal relatively easily, depending on exactly how much type information there is, how difficult / possible it is to correlate that with the AST.

Scala 3 is doing the typed AST thing, so hell, why not?

<comment><USER21> > I think that ruby 3 could lend itself to bring transpiled to crystal > relatively easily

I think for this to be tractable you'd basically have to implement a complete Ruby interpreter in Crystal and then transpire the Ruby program to bytecode to be run by the Crystal interpreter. So not really a meaningful or useful transpilation any more, and certainly not fast.

Even basic things like method dispatch do not have the same semantics in Crystal as in Ruby, so almost nothing could be tarnspiled 1-to-1.

<comment><USER20> We don't know very much about what ruby 3 types will look like or what other information we'll have.

Even a small subset of ruby being transpiled would be a useful thing for some developers.

You're much more aware of some of the constraints here than I am, as Oracle doesn't pay me to hack ruby for a living.

With a typed AST, there'd be enough information there to build a foundation for a rb2cr tool. I didn't say the two languages are best friends and it'll be a breeze.

Most of the ruby code I want to rescue isn't littered with string-based define_method nonsense. Most of ruby I think worth saving at all is outside of rails and exists in various tools or maybe some metasploit modules or stuff like that. The pieces of code that utilize the grossest dynamic elements of ruby, like define method or objectspace, that stuff doesn't need to survive.

Even getting 50% of a codebase in ruby compiled to reasonable crystal is a much better foundation than previously purported successors (elixir, scala, swift) can do.

If ruby gets a typed ast, I'll try and write a simple transform tool for the simplest ruby.

<comment><USER21> > Most of the ruby code I want to rescue isn't littered with string-based > define_method nonsense.

I see what you are saying - but this is where the issue is I think. You may not write code that does sophisticated metaprogramming, but the gems you use are probably fundamentally based on it. Even just requiring some of the standard library uses a surprising amount of of metaprogramming. For example you can't load something as basic as 'fileutils' without doing a lot of metaprogramming.

The entire Ruby ecosystem is built on metaprogramming.

<comment><USER20> That's where the boundary for transpilation will lie then, but I don't consider that to be a good enough reason to justify not trying to write a tool like that.

I really appreciate your thought on this! I look up to your work a lot. Sometimes I wish I had spent my career with compiler a instead of disassemblers...

Yes, there will be a lot of ruby that can never by crystallized. Yes that kinda sucks. So yes it's very good to explore optional or gradual for incremental refactors and modernization. But if even 1% of ruby code CAN be crystallized, then it's worth it to do it. Maybe not for you, but my time isn't nearly as expensive as yours must be, so I understand your reasoning here I think.

I don't mean to be adversarial here.

<comment><USER21> Part of the reason it might be coming across as a slightly negative reaction is that I'm passionate about implementing Ruby exactly as it is. If people want to write in Crystal that's great it can be fast. But if people want to write in Ruby, doing all sorts of metaprogramming, or that's the code they actually have today and need to run in order to keep their business going, then I want to make that just as fast for them automatically. Without telling them to use a subset, or not to use awkward features. I want people to bring me their insane code and I'll find a way to make it fast for them. That's the challenge I'm enjoying at the moment.

If people are happy to use a subset of Ruby then yes it could be transpiled. But a simple subset of Ruby should work great in the new JIT compilers we're getting anyway, without using Crystal.

<comment><USER20> I used to be happy with Ruby, but learning Scala and then Crystal really changed my mind about where the line ought to be drawn wrt to expressiveness and safety.

I don't and have never used the vast majority of Ruby's metaprogramming features, but I'm the odd duckling in an equation where probably 95% or more Ruby programmers are Rails programmers, and I am not. I mean, I can, but I don't. That factor makes less sympathetic to the majority of Ruby users who won't have any stake in a project like what I'm thinking about. And that's ok. And either way, if they want to make use of all those features then they will or maybe already have realized that Ruby already does what they want and Ruby3's care for backwards compat surely will cater to that majority in order to keep them on board. Metaprogramming I think is the primary style of Ruby, and that's OK.

The Crystal team has essentially reprogrammed the way I think about how it should feel to write good OO code. Scala did the same thing for a time, but just the sheer pain of SBT drove me away. With Crystal, Having parametric modules is an incredible advantage. I can write mixins essentially like traits that specialize on some new type in my program.

I have to consider how my methods end up typing out, and that changes the way I think about how I'm going to get from here to there. With Ruby, I really like that left-to-right idea of chaining until I arrive at the structure I want, and the more Crystal I write the more I realized how irritating it was to hunt down Ruby bugs by:

whatever.tap{ |o| puts "#{o} is a: #{o.class} here" }

or even deeper with responds_to? or whatever I'm trying to figure out. With crystal, all I have to do is look at the stacktrace, it is getting a nil where it shouldn't be at SomeClass#some_method on line 230842349, and I immediately can work on the bug at the source of it, instead of the tedious extra step of finding it.

Being able to make Ruby do magic was never part of the equation that held me a romantic captive to Ruby, it was always the succinctness. Elixir came close, but Crystal gets me all the way there.

If nothing else, maybe I will learn something new trying to find a subset of ruby3 that I can transpile.

I haven't read much about the new JIT, but if you're happy about it, maybe I should stop procrastinating and find out about it.

</comment></comment></comment></comment></comment></comment></comment></comment></comment><comment><USER22> There's so much unanswered here, does it handle parametric polymorphism? Does it allow sum/product types? Is there inference? Can you bound generic types?

IMO it would be a complete waste of time to add a type system that only prevented the most trivial of type errors: mismatching Integer and String.

</comment><comment><USER23> Interesting choice, considering that Stripe is a massive Scala user.

Is it really cheaper to write type checking for one of the most dynamic languages ever, than port to their other language? Seems to be strong evidence against the "stack doesn't really matter" viewpoint.

<comment><USER24> Yes it is. Stripe can do this with three (badass) engineers while the hundreds of other engineers continue to build their applications in Ruby. Stripe's Ruby code base is far larger than their Scala codebase, with each being used for the types of uses that play to their strengths. Rewriting millions of lines of Ruby into another language would more or less involve pausing all application development for one to two years, which simply is not feasible.

This way, a small team of skilled engineers make this typer work, and guide its general adoption, getting the codebase well typed over the course of years, all the while Stripe continues to grow its product suite and further conquer the world of e-commerce.

Twitter did a huge Scala rewrite. There are many lessons to be taken from their lack of product velocity while doing so that the Stripe team took onboard when making this decision.

</comment></comment><comment><USER25> ok this is going to sound snarky but it's an honest question: Why not just use Crystal[1]?

It's basically just compiled ruby with static types, and it can infer types correctly in most cases without you explicitly noting which they are.

The best argument for this (instead of Crystal) is keeping access to ruby gems.

[1]: [https://crystal-lang.org/](https://crystal-lang.org/)

<comment><USER26> Is Crystal a superset of Ruby where any valid Ruby project is a valid Crystal project -- just lacking type definition? If Crystal can do that then I think adoption of Crystal would benefit greatly.

<comment><USER25> As @chrisseaton said, no, BUT some ruby is valid Crystal. I think it would be more accurate to say that a subset of Crystal is valid Ruby, and a subset of Ruby is valid Crystal.

<comment><USER27> That's correct. You can run your Crystal programs with the Ruby interpreter, but you can't compile your Ruby to Crystal (simple scripts usually work, but bigger things that use use gems probably won't).

<comment><USER28> You can't run a lot of Crystal code in Ruby, either. They implement keyword arguments completely differently (Crystal does it more like Python, where all arguments can be specified by name or position) and of course Ruby doesn't support Crystal syntax relating to types. Crystal also has a tuple type with a literal syntax ({1, 2, 3}) that isn't valid Ruby. But otherwise it's fairly close.

<comment><USER27> Thank you for the correction, I was not aware of this.

</comment></comment></comment></comment></comment></comment><comment><USER29> This Ruby typechecker is a similar approach to Facebook's "Hack" language, incrementally upgrading their legacy (PHP) codebase to a new statically-typed language instead of rewriting everything in a new language.

[https://code.facebook.com/posts/264544830379293/hack-a- new-p...](https://code.facebook.com/posts/264544830379293/hack-a-new- programming-language-for-hhvm/)

</comment><comment><USER30> Putting this out here in case if anyone finds it useful - I like a lot of good ideas from Contracts.rb such as the type signature looking more expressive (personal taste - looks Haskell/Elm ish) and support of Maybe's. Granted it's not static but def some good ideas to steal from.

[https://egonschiele.github.io/contracts.ruby/](https://egonschiele.github.io/contracts.ruby/)

</comment><comment><USER31> Typical Rails codebases already enjoy a reasonable layer of 'typing' via ActiveRecord coercions, validations, and possibly some system that declares/documents the types/structure of your REST API.

Also, I advocate keyword arguments (especially 'required' ones), which tend to kill 80% of the use case of typing for dynlangs ("what's the type of argument x?")

A meaningful test suite would hammer the last nail in the coffin.

</comment><comment><USER32> Is the idea that you would take your existing code and add typesafety to it? A statically typed language is faster because it does type checking at compile time and does not need to do "if string, ok else throw exception", so adding a bunch of type checking to a dynamic language adds the development overhead without the benefit. I suppose there is a false economy in that statement as defects caught early save time vs late defect discovery, which in my experience, is very common due to type issues in dynamically typed languages.

I personally prefer working with statically typed languages but I've been working with elixir for the last 8 months or so and do not feel very compelled by the use of dialyze (type checker) in the dynamically typed language - <USER2> get used to the idioms and try to keep the code type-y but it seems unidiomatic to take type safety too far in a dynamically typed language when you don't get the performance benefits. You cover the things you're likely to get wrong but embrace it for the sake of development speed otherwise. For a new project the right question to ask is "why would we use this instead of a statically typed language." I suppose for an existing project the question this is trying to solve is "how can we reduce bugs now that we've made it to market."

</comment><comment><USER33> I was at RubyKaigi. This presentation was one of the best delivered, so kudos to the Stripe team for all the work put into it.

A question I didn't get to ask while there is -- will there be some sort of type sharing system introduced?

I've been a user of both Typescript and Flow (both great tools), but saw Typescript's popularity soar because of community written types.

<comment><USER34> > A question I didn't get to ask while there is -- will there be some sort of > type sharing system introduced?

Yes. This was one of goals of our talks. We wanted implementors of type systems for ruby to start collaborating, in particular on a repository of typed shims.

The details of how this would work out might be different. Typescript and Flow change syntax and thus they cannot be included inline in arbitrary JavaScript libraries. We're intentionally compatible with Ruby syntax. Thus we would like this "repository of types for libraries" to only contain types intermittently, until types have been accepted to upstream of respective library.

<comment><USER33> > We wanted implementors of type systems for ruby to start collaborating, in > particular on a repository of typed shims.

I see. Do you see this being a mono-repo of types that's under a Sorbet org or something Sorbet looks for in a gem if provided? Opening issues for a project's types with Typescript has been somewhat painful since many live in one repo [1].

1 - [https://github.com/DefinitelyTyped/DefinitelyTyped](https://github.com/DefinitelyTyped/DefinitelyTyped)

</comment></comment></comment><comment><USER35> Is there any discussion on why Stripe chose to write so much code in Ruby? I would have thought for financial software where subtle bugs can have a huge impact you would want to opt for a language with static checks. Or maybe only a small part of the codebase requires this?

</comment><comment><USER36> I have to say, after using Stripe for a few years - I'm amazed at how well it works. In fact, I can even create a "refer a friend" system on a rails app in some 15 lines of code[1].

I'm excited if they can bring the same simplicity to their additions to languages and frameworks as well.

[1] [https://blog.projectpiglet.com/2018/03/refer-friend-using- st...](https://blog.projectpiglet.com/2018/03/refer-friend-using-stripe- rails-15-lines-code/)

</comment><comment><USER37> Interesting! Anyone with insight into the ruby core team know if Ruby may add typing (optional or mandatory) in the future?

<comment><USER38> Was at the Kaigi and on the opening keynote Matz talked a bit about it. He doesn't really support adding it since he feels like in 20/30+ years, advances in technology will make writing types "obsolete".

<comment><USER39> It's a weird argument. If it's not necessary in 30 years (I doubt it because some of the type inference issues are simply undecidable, and that in much simpler languages already), then the language can just opt to ignore type annotations or merely check their consistency. Having them is great either way for the benefit of documentation.

My take is he doesn't really like (or "get") types and feels protective of his language, which is fair. I think Python made an interesting choice here, i.e. provide syntax for types without semantics. The drawback there is potential fragmentation on type system semantics.

</comment></comment></comment><comment><USER40> So, Sorbet is to Ruby what Dialyzer/Dialyxir is for Erlang/Elixir right?

Are they using basically the same approach?

</comment><comment><USER9> Having done some legacy Ruby work I'll say this is sorely needed.

</comment><comment><USER41> Is this analogous to mypy?

<comment><USER11> It is a similar idea, yes.

</comment></comment><comment><USER20> I think the most fascinating thing that Ruby 3 could approach is an approach similar to Dart 1's optional types. That way a fast unchecked mode could be available for developing, and REPL usage, and then a separate switch could be flicked for building a release.

Seeing the Ruby devs take compatibility so seriously is such a confidence builder for me, either way. Ruby is such a joy to program in, and for certain projects, Crystal fits the "square enough to go in a square-ish hole" role, but I really miss some of the ease-of-use that Ruby has to offer.

require 'Something'

Something.new.methods.sort

I use that every single day. Crystal can't do that. Yet.

Little things like that make Ruby such a human-friendly language.

Now that we've learned our collective lesson that naive dynamic types are too slow for big projects, I am glad to see Ruby-thinkers begin to explore the type system space and search for something that will benefit all Ruby hackers.

I suspect that if we arrive at the point where Ruby has a gradual or optional type system available, that refactoring some of the old Ruby tools will be very pleasant. If or when that day comes, I know I'll devote a chunk of my own time to try and catch the tools back up, and the rest will follow if only there are hackers who want to hack badly enough.

<comment><USER42> A lot of people that love Ruby and dislike Crystal seem to dislike Crystal for reasons like what you mentioned. I'm pretty sure that you can get a sorted list of methods in Crystal (not with the same syntax unfortunately) at compile time in macros. I'm not sure why you would ever want to dynamically get a sorted list of methods at runtime... unless you're adding methods at runtime.

<comment><USER20> I use it in the repl for classes I'm not familiar with. I use Ruby for duct taping a lot.

</comment></comment></comment><comment><USER43> People created dynamic languages for the exact reason of not having to define types.

Now the trend is to bolt on type checking to dynamic languages because people don't want dynamic behavior. Most developers I've encountered who seek to add types to dynamic languages will never truly understand dynamic languages, closures, concurrency passing or functional programming for that matter.

If you want a statically typed language, use one. Use one that was designed for deterministic behavior and static correctness.

If you have a huge codebase in Ruby that is full of bugs and is unpredictable, don't blame the language or lack of types.

There are plenty of ruby codebases that are bug free and work well without static typing. How do they exist?

<comment><USER44> > Most developers I've encountered who seek to add types to dynamic languages > will never truly understand dynamic languages, closures, concurrency passing > or functional programming for that matter.

I don't see how seeking to add types precludes also understanding "closures, concurrency passing or functional programming", especially since these are in no way foreign concepts in statically typed languages.

> If you want a statically typed language, use one. Use one that was designed > for deterministic behavior and static correctness.

I definitely see what you're trying to say, but as I understand it, these projects to "add types" often come on later, when the project is so big, that it is simply less time consuming to just develop a type checker, than it is to rewrite your entire codebase in a new language.

<comment><USER43> My argument is that you can always implement your own breed of type checking either via convention (with appropriate enforcement via static analysis or linting tools if so desired without having to run the code), or you can do type checking at run time -- most dynamic languages have a facility to interrogate a variable to determine its contents and you can implement any number of tests on it as you see fit to validate it.

<comment><USER45> In practice the difference in guarantees between a static type checking system and a runtime one are so vast that I think they're mostly incomparable. Runtime checking is like doing most of the work of modelling a program with static types but achieving a fraction of the benefit.

</comment></comment></comment></comment><comment><USER46> I don't understand the appeal of static typing. It always feels like I'm adding overhead without receiving any benefits. I've worked in some fairly large dynamically typed codebases and have never run into issues with type errors. Static typing does not alleviate the need to test your code, which is a more effective way of reducing bugs in your system than annotating your methods/functions.

<comment><USER47> Not sure I've ever heard anyone say static typing removes the need for testing.

Types are self-documenting. Onboarding new devs in a _large_ static codebase will be 10x easier than a dynamic one.

Coding is actually faster in statically typed projects - autocomplete works every single time and you never have to dig into another source file to see what objects/methods you have.

And, most modern type systems (Typescript, Swift, etc) don't have much overhead at all with their impressive type inference.

<comment><USER48> It's a double edged sword - onboard some devs on "advanced" scala code and let me know how the 10x is working for ya. I also feel that sometimes types cause additional congitive load - you have to mentally model the types and what they do. OTOH I'm pretty sure most people can onboard a js project that's well written.

I think exploratory, debug-based onboarding can be easier with poorly written static lang codebases that dynamic, but then again depends on what convetions are used. I don't care much about autocomplete unless I have to type ReallyLongClassNamesThatAreSupposedToBeMeaningful; besides coding is mostly about avoiding errors, not typing faster.

In the end, whatever floats your boat but dyn langs surely have their place.

<comment><USER47> I've heard the "mentally model" argument a few times, and tbh I don't quite understand it. You have to model the object regardless, either with types or tons of extra checks to make sure that object has what you need. I've seen so much javascript code do exactly that because the methods had no idea what they were going to be given. The contract of static typing eases that load, I think.

<comment><USER49> Ya for sure. Without getting a debugger into the application code, it sometimes seems completely impossible to look at a piece of dynamic code you're not already familiar with and decide if it's going to work. I've found this especially tough during code reviews for new stuff that make heavy use of abstractions, even when the abstractions are good.</comment></comment></comment></comment></comment>


'DOM Doodles' from actual interviews at Monetate - <AUTHOR> http://engineering.monetate.com/2012/04/23/dom-doodles-from-actual-interviews-at-monetate/

<comment><USER2> That looks like fun!</comment>


OS X Snow Leopard vs. Windows 7 - <AUTHOR> http://www.theonion.com/content/infograph/os_x_snow_leopard_vs_windows <comment><AUTHOR> IMO, a fine piece of humour: on the surface it's a simple gag, but there's enough insight to make you think. Matt Maroon will like point three.</comment>


Facebook retargeting - <AUTHOR> My husband was looking at an Airbnb on his phone (we were thinking of possibly staying in that area but hadn't booked anything yet . He had their website open. He read out their details to me. We discussed it. I was looking at my Facebook feed at the time and an ad for that particular Airbnb rental appeared immediately on my Facebook page. Very unnerving!I have never looked them up or viewed their website! How did this happen? Very creepy! <comment><USER2> Here's a solid debunking of that theory:

[https://www.wired.com/story/facebooks-listening- smartphone-m...](https://www.wired.com/story/facebooks-listening-smartphone- microphone/)

It's more likely that you are being targeted with "lookalike audience" advertising like this:

[http://www.geofli.com/blog/target-your-users- strategically-w...](http://www.geofli.com/blog/target-your-users- strategically-with-ip-address-in-facebook-ad-manager/)</comment>


When is a Startup ready to start growing their audience and user base? - <AUTHOR> Hi all,
I was here thinking, when is a Startup ready to start growing their audience and user base? After their seed round? After more mature rounds like Series A?
Until then what do they do? Product development and talk to potential users?
I was wondering, not all startups are equal. Some get out from YC ready to explode, others keep on stealth mode some more months/years, and others simply die.
What do you think? <comment><USER2> Don't wait, the best problem is being underprepared for unexpected growth</comment>


Louisiana records give insight into businesses that utilize prison labor - <AUTHOR> https://www.muckrock.com/news/archives/2017/jun/08/Louisiana-inmate-labor/ <comment><USER2> Sounds what you get if you sell what essentially are sovereign duties of a state to the highest bidder: An extremely lucrative system of modern-day slaves that have to shell out their hard earned below-market dollars to a rigged monopoly system completely provided by my buddies (of 10$/min landline calls, cash transfers with 50% fee, all kinds of shady "service charges") to extract the maximum amount of cash while they're in.

So what do you do if you're sitting on such a gold mine? Well, I'd probably fuel an endless and hopeless war on drugs, introduce mandatory minimum sentences and take a really, really tough stance on those immigrants. Keep 'em coming.

Completely coincidentally, the US has the highest incarceration rates on any halfway civilized nation on the planet by a large margin.

Free markets for the win.

<comment><USER3> When you work for no pay in a communist prison camp, we call it a slave labour camp. And why wouldn't we? There are no slaves in America.

When you work for next to no pay in an American prison (And are then billed for room and board when you are released...) Well, that's something else.

How many millions of lives has this institution of liberal democracy destroyed?

<comment><USER4> > How many millions of lives has this institution of liberal democracy > destroyed?

How is this an institution "of" liberal democracy? I can either interpret that as liberal democracy as destroying lives, or that working while incarcerated is a feature of liberal democracy. I don't think either are true.

<comment><USER3> Liberal democracies get extremely holier-then-thou about slavery - state- sponsored or otherwise. Yet, they have no natural means of preventing it from happening inside them!

My point is that we turn a blind eye to it happening at home. People working in the GULAG were also paying their 'debt to society'. Not all of them were political prisoners, either.

<comment><USER4> So is the "institution" the different perception of local incarceration vs incarceration in other countries, or the incarceration itself?

<comment><USER3> It's a difference in perception - cognitive dissonance. There are obviously differences in the incarceration (Our prisons ruin many lives - but Soviet ones ended them. Depending on the year, mortality rates were between 1 and 20% - the highest were during the war.)

<comment><USER4> Ah. The original comment seemed to imply it was the prisons that were a feature of liberal democracies, which seemed a weird leap to take. I agree with you in general. The penal system is just one of many systems that's overly politicized in a national context. It's part of the general problem of the U.S. not being very good at relative comparisons with other countries without reverting to a nationalistic dick-measuring contest.

It's the nasty downside to American Exceptionalism. On one end, it spurs people to attempt to be better because they expect they _should_ be better, on the other it causes people to infer differences between our attributes and those of others that we don't particularly like, even when in practice those differences don't amount to all that much.

</comment></comment></comment></comment></comment></comment></comment><comment><USER5> to me one of the most interesting things about this issue is how it trended recently because someone actually read and thought for a moment about what Hillary Clinton wrote about in her best-selling book "It Takes a Village", about how they used prison labor during her time in the Arkansas governors mansion:

[http://www.newsweek.com/hillary-clinton-prison-labor- african...](http://www.newsweek.com/hillary-clinton-prison-labor-african- americans-arkansas-622209)

the fact that this was only widely noticed 20 years after publication is amazing

<comment><USER3> This really needs more recognition. The fact that this practice is so normal is appalling.

<comment><USER6> I agree.

I think this story is an enormous opportunity to get bipartisan opinion against abuse of prison labor. What better way to get popular support against AG Session's incarceration increase than to tie it to a deeply unpopular politician with his constituency?

<comment><USER5> i see you deleted your previous comment saying that the way this 20-year-old passage has suddenly been noticed is "banal" and passage itself is not newsworthy, because HRC was not newsworthy anymore until she ran for president.

the point of my observation is that this passage only became a widespread topic AFTER the recent election, after a sharp-eyed and thoughtful reader commented on it via Twitter and it went viral last week.

<comment><USER6> Yeah, I deleted it. I was agreeing with you. The comment was focusing on the circumstances, when the value is recognizing the opportunity.

</comment></comment></comment></comment></comment><comment><USER7> It's not obvious to me that prison labor is immoral or equivalent to slavery. As long as the working conditions are humane and the garnished wages are principally used to subsidize the cost of their incarceration, I see no issue. Personally, I think prison policy improvements should start with common ground (e.g., guards shouldn't beat prisoners, prisoners should get prompt medical attention, etc) before we start worrying about issues that boil down to prisoner comfort.

<comment><USER8> Regardless of what you want to call it, it is forced labor, which is still a definition of slavery, even if it is a more expansive one.

<comment><USER7> Fine, call it what you like, my point is that it's not the moral equivalent of conventional slavery.

<comment><USER3> It's not chattel slavery, but 'conventional slavery' doesn't have to be chattel slavery to be slavery.

Across history, there have been many forms of slavery that were not chattel slavery. This is one of them. They have all been morally repugnant.

<comment><USER7> Prison work has never been widely regarded as morally repugnant (don't conflate with debtors' prisons, which are fundamentally different), so this necessarily is not one of your aforementioned forms of slavery.

At the end of the day, incarceration costs money, and the people responsible for the incarceration would ideally foot the bill. Obviously the incarcerated rarely have such means, but I don't think it's unreasonable to ask them to pay a little bit more than the average law-abiding citizen. I don't have a strong opinion about how they repay their debts--whether it's garnished wages while in prison or after their release, but it should be one or the other. Actions have consequences, and a fundamental purpose of prison is to remind people of that. Moreover, work is a rehabilitating endeavor--it build skills, confidence, and character. It's mutually beneficial.

</comment></comment></comment></comment></comment><comment><USER8> At a minimum, the businesses should be paying the state the prevailing wage for the jobs being done (the business owner should not be a beneficiary of below market labor).

Even better if the prisoners keep a substantial portion of that.

<comment><USER9> A bunch of this is for the "transitional work program" which sounds more like a way for prisoners to get out of prison more smoothly: [http://www.doc.louisiana.gov/transitional-work- program](http://www.doc.louisiana.gov/transitional-work-program)

I don't know anything in particular about the program, but why do you think the business should pay the prevailing wage? How many prisoners would get hired with that requirement vs. without it? As a business owner why would I prefer to hire a prisoner instead of someone not in prison? Or am I thinking about it the wrong way? It sounds like hiring a prisoner benefits the prison system and the prisoner (and also the business as long as there is an incentive of some kind).

<comment><USER8> Because it puts the business owner ahead of their competitors and neighbors (they are gaining from the favor of the state).

You make a fair point of incentives to hire prisoners rather than others though.

<comment><USER9> There are going to be costs, but maybe this is actually a relatively low-cost option for helping to get prisoners back out into the world (and not back into prison). It is certainly cheaper than keeping them in prison. It says on that page that 10-20 percent of prisoners stay with their employer after being released. That seems low to me, but I haven't studied other solutions to this problem for comparison.

</comment></comment></comment></comment><comment><USER10> It's an interesting breakdown by job. Most of it's just "general labor," but I'd love to hear the story about the person doing forced prison labor as a caterer, or even better, the person doing forced prison labor as an offshore oil platform engineer.

<comment><USER8> A lot of "interesting" jobs are for the state itself:

[http://www.nola.com/politics/index.ssf/2017/05/prison_inmate...](http://www.nola.com/politics/index.ssf/2017/05/prison_inmates_politicians_min.html)

</comment></comment><comment><USER11> I just reviewed the wages vs min wage and i dont see how this can be true, unless these are trusted prisoners who work off-prison jobs.

Prison jobs are notoriously low paying such as cook or dishwasher which ranges from $0.10-0.15/hr.

I interviewed several prisoners at a state prison in utah (draper perhaps?) back in 2005 and got the scoop and it was very belittling to see that prisoners were paid that low.

i just dont see how it could've jumped so high in only ten years.

From the info i gathered this is not a utah phenomenon either the prisoners are paid extremeley low wages in other parts of the states as well.

</comment><comment><USER12> For those calling this "slavery" thinking that inherently makes it illegal, I quote the 13th Amendment: "Neither slavery nor involuntary servitude, except as a punishment for crime whereof the party shall have been duly convicted, shall exist within the United States, or any place subject to their jurisdiction."

Not saying I agree with the practice. But just getting other people to agree that it's slavery won't change anything.

</comment><comment><USER13> Is this part of the same complex that runs prisons for profits, or a mixture of government operations and private companies?

</comment><comment><USER14> It's definitely an improvement compared to the State of Louisiana selling children born to female prison inmates to bidders / Slave Owners on the steps of a Courthouse before the Civil War. Yes, this actually happened. About 10 times over the course of a few years. Source: I knew the guy who did the research and published the findings.

<comment><USER15> > Source: I knew the guy who did the research and published the findings.

That's not really a source. If this was published, then tell us where! The published findings would be the source.

</comment></comment><comment><USER16> slavery</comment>


The weirdest thing women say to me - <AUTHOR> http://swizec.com/blog/the-weirdest-thing-women-say-to-me/swizec/6450 <comment><USER2> I don't have a problem talking to people (I'm a software engineer who happens to have a Bachelors in Theatre and a Masters in Education), but I do understand the social frustrations. For someone who thinks in terms of code and logic and problem solving, decoding human interactions is tremendously difficult. In mathematics, there is often one correct answer -- or, at the very least, a discrete set of correct answers.

In the real world, when a girl smiles at you, what does that mean? It could mean anything.

It could mean she thought of something funny. Or I look funny. Or she's just happy. Or she's flirting. Or she likes me.

For someone who thinks in terms of information and problem solving, being "outside" is really, really hard work. It's hard work to talk to people. Sure, some of us get the hang of it, and you'd never really know that we're uncomfortable, but if the social interaction goes on long enough, we tire out. We fall back into our solitude. It's easier that way. It's less work. Those correct answers we're trying to find matter only to us -- if we get it wrong, nobody else has to know.

In my experience, the problem with social introverts (and engineers in general, though they are not necessarily the same thing) is that social interaction is exhausting, and because of that, most of them just never learned to fake it.

It's unfortunate. But the best thing you can do for an engineer is let them talk about what they wanna talk about, act positively, smile, and actively invite them out again next time. And if they say no, it's probably nothing personal. They're just tired.

<comment><USER3>
      "most of them just never learned to fake it"

This brings out an important point. Maybe engineers are so used to being forced to face the cold, scientific truth everyday (especially in programming) that they despise a false face or show of excitement. Salesmen on the other hand...</comment></comment>


Analysis of the conflict among Canonical, GNOME, and KDE - <AUTHOR> http://blogs.gnome.org/bolsh/2011/03/11/lessons-learned/

<comment><USER2> The key problem is that the desktop community desperately needs one or more architects who can act as a sheepdog and keep the sheep marching in roughly the same direction.

It's far more critical in the Desktop world than in the Linux kernel because in the kernel, there is enough development in the kernel that there is strong incentive to develop in the upstream sources, and not behind distribution- specific source trees. In addition, the Linux kernel releases on a regular 3 month cycle. So even if we don't have formal design documents, it's a lot easier for collaboration between key kernel subsystem developers, because our patches have to come together and _work_ every three months when we push out a release. (In practice, they had better mostly work by the end of the two week merge window, or there will be hell to pay, and Linus, as the great benevolent dictator, will yell at people, and Make a Decision --- which will usually involve reverting the change which broke things functionally.)

In contrast, the Desktop world has a much harder problem, because (a) not everything lives in one source tree, (b) the kernel largely doesn't have a UI, and there are objective measures of "does it work", and "did the benchmark results go up or down", that either don't exist or aren't as important in the UI world. Questions of whether a UI is "good" or "better" than some other UI choice is a question which can't be easily answered objectively. And that, more than any thing else, is probably why we have three major design centers for desktop development: GNOME, KDE, and Canonical.

One of the things that worries me is that all too often, people look at the way things are done in the Linux kernel, and assume that it will port over to their project. In most cases, the Linux kernel is a much larger and more complex project, and things which we do in the Linux kernel world probably aren't needed in other projects. But in the Desktop world, I think the reverse problem is true. It's a much larger and more complicated ecosystem than what we have in the Linux kernel, and so decisions such as allowing the major subsystem developers to negotiate major technical changes organically and informally might work for Linux, but not in the desktop world. (For example, in the storage stack, there are less than a half-dozen people I would need to consult with in order to come consensus about some change to accomodate PCI- attached flash, and we have multiple opportunities a year to get together and work things out). Hence in the desktop world, hiring a full-time architect who did nothing else but talk to the different stakeholders and try to come to consensus on a common vision is probably something which is necessary, even though we haven't found the need to have someone in that role in the Linux development community.

</comment><comment><USER3> Posts like these really show the problems in the current free software development world; balkanization between different stakeholders and bizarre processes that alternate between bureaucracy and anarchy. Watching the tug-of- war between Red Hat and Canonical play itself out in the form of dysfunctional community development process is depressing. I'm aware that watching sausage being made never makes you very hungry, but as a user of free software, I don't feel my needs are served very well by the territorial and political battles being fought.

What makes it even more frustrating to me is that not only are KDE, Gnome 3, and Unity all failing to cooperate successfully on shared resources, but they are all marching resolutely in the wrong direction, regardless of the pleas of the user community. So far as I can tell, everyone hates Gnome Shell, everyone hates Ubuntu Unity, and everyone thinks that KDE 4 has been a big mess compared to KDE 3. I see all of these projects as engaging in a slappy fight as they march together off the cliff of unusability and misguided redesigns.

<comment><USER4> I think to be fair to all three, you have to give them six months after they are actually released before we judge them. As is typical in OS Linux community, a very vocal minority is probably speaking up and "hating" all three. I would wager that the majority opinion is much different than what you hear from the vocal minority.

<comment><USER5> KDE 4 was released in January 2008.

</comment></comment></comment><comment><USER6> As a KDE developer myself, I have to admit to being very disappointed by the very first section regarding the development of the StatusNotifier specification. Dave complains that there was no problem statement, points out a quote relating to D-Bus, and somehow still entirely misses the point even after claiming to have read through the threads relating to the proposed spec.

The problem statement is _easy_ : Have a way of separating notification items and applications, such that the notification item is part of the desktop and not part of the application. In other words, XEmbed sucks as a way to have notification items in a system tray. Anyone who has ever used e.g. GAIM in a KDE 3.5 desktop with a pixmap background knows exactly what I'm talking about, as GAIM's icon would show up as an ugly box with an icon inside, while the KDE application's would have a notification item with a properly transparent icon drawn.

Claiming that the entire idea was merely to hook up notifications into D-Bus is so far afield that it mystifies me completely, and claiming that _no one_ has ever had issues because of this is patently false. In fact, the __VERY EMAIL __[1] that Dave quotes has an _actual problem statement_ directly before it:

<quote> In the past few months in KDE we worked on a new way to represent the systemtray icons to overcome the following limitations:

-lack of communication between the systemtray area and the items, that mean we don't know about their status, their importance of if they are being used or not

-the xembed process is quite slow and doesn't give control to the systray on the paining

-it's not possible to have more than one systray (useful in multi monitor setups)

</quote>

I know this because I actually bothered to use GTK+ and GNOME software even with my KDE desktop, but now I wonder if Dave has ever tried the opposite.

I also was passively watching the threads as they happened back in the day, and stating that there was disagreement due simply to lack of a problem statement would be flatly false.

Dave then goes on to accuse Aaron Seigo and Marco Martin of being obstinate in their positions by that point, which simply does not mesh at all with my memory of things. For instance, A GNOME Panel hacker named Frederic Peters mentioned that it would be nice to be able to merge launchers with notification icons [2], an idea that was warmly received, not thrown out [3].

In addition, given that the implementation changed several times during just the KDE implementation period, there would have been no major problems with even incompatible changes to fix problems. For instance Aaron Seigo in reply to Dan Winship recommending a different way to send notification signals over D-Bus mentioned: "yes, this is a nice idea for limiting the bus traffic. would require an incompatible change to the spec, but it is doable." [4]

I'll read the rest of the article, but if this is the kind of "uncontroversial" fact stating that's going to be going on then I'll probably just unsubscribe from the xdg list and get on with my KDE developing life...

[1] [http://lists.freedesktop.org/archives/xdg/2009-September/011...](http://lists.freedesktop.org/archives/xdg/2009-September/011038.html) [2] [http://lists.freedesktop.org/archives/xdg/2009-September/011...](http://lists.freedesktop.org/archives/xdg/2009-September/011072.html) [3] [http://lists.freedesktop.org/archives/xdg/2009-September/011...](http://lists.freedesktop.org/archives/xdg/2009-September/011073.html) [4] [http://lists.freedesktop.org/archives/xdg/2010-January/01122...](http://lists.freedesktop.org/archives/xdg/2010-January/011229.html)

</comment><comment><USER7> These problems might take forever to solve [as the organisations are large]... In the meantime, I wouldn't be surprised if someone came up with a better technology model as a replacement.

Id like to see a desktop development model more like modern web development - UI in HTML5/javascript/SVG, json data messaging, UI & system events coming from a port [eg. shinetech/eventserver], able to write small server-like plugins in Node.js / C / Ruby / lisp that can provide services and be remoted away from the UI.

Would be nice if some of this was reusable across OSX/Win/Linux. Perhaps my desktop work should be done in the browser?

The old unix guys might have approached it this way if they had the current web at hand to reuse/augment/embellish upon. Maybe this is more in the true spirit of unix than QT/GTK/Xwindows?

<comment><USER7> Can anyone point me to a good architecture/dev overview of Gnome Shell and Unity ? I find it hard to sift out the marketing from the tech in what Ive seen.

<comment><AUTHOR> Unity is a plugin for the compiz window manager.

<https://lwn.net/Articles/430686/> <https://wiki.ubuntu.com/Unity> [https://wiki.ubuntu.com/Unity?action=AttachFile&do=get&#...</a>

<comment><USER7> thx, I do trust LWN to give us the gist.

</comment></comment></comment></comment><comment><USER8> I'm interested to see how this ends up being resolved (I use Ubuntu, don't like Unity, and Gnome Shell looks to be just as bad) but have only been half- following this drama. This is a really great summary post.

I'm totally unsurprised that Shuttleworth's penchant for decision by fiat and Jobsian cargo-cultishness has driven a wedge between his company and others in the FOSS movement. (For example window controls, before windicators were announced: "Yesterday I decided to move window controls to the other side for our LTS. It's for a secret project. DO NOT ASK ME WHAT THE PROJECT IS."). But I'm also unsurprised that disorganization and what also appears to be at least a little ego and maybe even jealousy has warped things on the other side as well.

</comment><comment><USER4> "My understanding of GNOME is this: GNOME does not have technical leadership – it hasn't had clear technical leadership since, as I understand it, the creation of the GNOME Foundation (at which point, by design, the board was given a mandate to build and define GNOME, and then soon afterwards removed that mandate from itself). The foundation does not now dictate any vision or direction for GNOME."

This is a huge problem, as I see it. Even in OSS, where you have very capable people, someone needs to provide direction. I'm reminded of a military story, which I will paraphrase.

Single most important thing in warfare is to have a known objective. That way if the person in charge is killed, as is the second in charge, the troops still know they need to take that hill or everyone dies.

I take the same approach to software and it has worked my entire career. If Gnome doesn't have a stated vision/objective, it is no wonder it looks like chaos to someone outside!

Someone from Gnome _should_ step-up and provide some leadership, some technical direction, some market direction or some general foresight that things are going in the right direction. I hope the author is wrong and there is someone already doing this, but if not...

</comment><comment><USER9> Great post. Excellent TL;DR summary, and well-articulated details on each point.

Hard feelings and lack of communication between competing UNIX desktop environments ... where have I heard this before?

</comment><comment><USER10> It's so sad that such great ideas are just so dysfunctional.

<comment><USER11> What is the great idea are you referring to?

(not trolling, I honestly don't know what it is that you mean)

<comment><USER10> The idea of free software. It's a vision the Open Source community has that all software should be free.

</comment></comment></comment><comment><USER12> I'm surprised that no one here has yet pointed out that posts like this are a part of the self-correction mechanism present in transparent communities. Sure, it may be drama in the short term, but I think the fact that we _know_ about these altercations is an important thing in and of itself. I would be highly surprised if this kind of drama didn't occur in the proprietary software world, but we don't get to see it and learn from it.</comment>


Reflections on Authoring a Minimum Viable Book - <AUTHOR> http://miningthesocialweb.com/2013/08/24/reflections-on-authoring-a-minimum-viable-book/

<comment><USER2> I've followed this development, and I love the use of Vagrant. The use of iPython is good too, now it would be nifty to extend it to use StarCluster and spawn off a host of cloud servers for an hour to chew through some big summaging.</comment>


Show HN: The Whicher: A/B test the Real World - <AUTHOR> https://www.thewhicher.com

<comment><USER2> This is way cool, though I can't help but think that the results might differ from actual A/B testing because the testing context is explicit. Have you done comparisons to backend A/B testing to see how well the two align?

It would seem that traditional A/B testing allows you to see what actually converts, while this framework would be biased towards user preference--which doesn't necessarily imply conversion. For example, I think Amazon's site is ugly and busy, and given the choice between that layout and a cleaner one, I'd probably choose the cleaner. That said, there's no way they haven't tested the hell out of the home page and discovered that a busy page, though uglier, converts better.

<comment><AUTHOR> That's true and that's part of the reason I built it. It's not intended to be a replacement for A/B testing, but a compliment.

A/B testing is great for lots of things, but it's not always the right tool for the job.

I use the term A/B testing loosely since it's obviously not actual a/b testing. It's a hybrid of a/b testing and a survey that allows you to ask a question an get much higher resolution, and test many more variation than a typical survey question by using some techniques from A/B testing

</comment></comment><comment><USER3> I know more than a bit about statistics and A/B testing. My first reaction is that asking for more information from a limited sample set does not actually give you the power to do more powerful statistics. Claiming otherwise is a bad sign.

Secondly a lot of what A/B testing gets at is subconscious reaction, not conscious reactions. Consciously when we're asked a question and are paying attention, we are poor judges of how we're going to react to, say, a larger button to click on. But when we're actually using the site, we notice and hit the larger button.

And thirdly, if you're actually going to seek out a small sample of people for detailed information capture, do it in person. I can't stress this enough. If you think up questions in a vacuum it is hard to ask the right questions. That is part of what makes A/B testing so frustrating. But if you have a conversation and are observant, you'll quickly generate good ideas.

<comment><AUTHOR> I agree that sample size is important and that's why you still need 200+ responses.

It's a hybrid between a survey and an A/B test. You ask a question, but you get to find out about more than just the one variation that the person is served up.

I think it's fair to say that there _is_ more info to be gathered from each individual than time-on-page, scroll behavior, and converted-or-didnt

If you like, you can think of this as a way to get more resolution from a survey-typed question

</comment></comment><comment><USER4> Interesting take on a/b testing. A small suggestion - it would be nice if clicking on the buttons after the text "Which icon do you prefer?" actually demonstrated how the service worked. It took me a while to realize that you need to click on the 1-2-3-4-5-6-7-8 steps to go through the process (they don't look clickable).

<comment><AUTHOR> Thanks for the feedback!

I have it auto-advancing from 1 to 2, which I thought would demonstrate that it was interactive, but I guess it still needs work :/

<comment><USER4> Yeah I noticed that. My immediate reaction was to click the buttons below the "Which icon do you prefer" question though, since that's how I assume the service would work. I didn't think to click the steps until after I had tried clicking everything else.

Could just be me though, maybe you should a/b test it ;)

</comment></comment></comment><comment><USER5> I like this hybrid of survey + A/B testing. Interesting approach. I agreed with marbemac, I also confused with the demonstration's UI on your homepage.

</comment><comment><USER6> Just checked out the site--love the survey-A/B concept. Is this a pivot for RootBuzz or another project?

<comment><AUTHOR> It's another project

</comment></comment><comment><AUTHOR> Hey guys, I created this tool and I'll be around answering questions throughout the day!

</comment><comment><USER7> A/B testing is very good.</comment>


Google's Android faces a serious Linux copyright issue - <AUTHOR> http://fosspatents.blogspot.com/2011/03/googles-android-faces-serious-linux.html <comment><USER2> _Google copied 2.5 megabytes of code from more than 700 Linux kernel header files with a homemade program that drops source code comments and some other elements, and daringly claims (in a notice at the start of each generated file) that the extracted material constitutes "no copyrightable information"._

Uh, what?

 _If Google is proven wrong, pretty much that entire software stack -- and also many popular third-party closed-source components such as the Angry Birds game and the Adobe Flash Player -- would actually have to be published under the GPL._

Stopped reading after this...

<comment><USER3> If you link to GPL libraries you must abide by the terms of the GPL. The same applies to using the kernel header files. Or at least Torvalds believes this to be the case, as do the IP lawyers quoted[1]. Quote from Linus:

 _"In short: you do _NOT_ have the right to use a kernel header file (or any other part of the kernel sources), unless that use results in a GPL'd program."_

 _"So you can run the kernel and create non-GPL'd programs […] BUT YOU CAN NOT USE THE KERNEL HEADER FILES TO CREATE NON-GPL'D BINARIES. Comprende?"_

The solution seems to be to re-licence Bionic under the LGPL, which is what the C libraries of every other linux based OS do. Although that does seem to suggest that one can always work around the GPL linking restriction by creating an intermediate LGPL library, but I'm no lawyer so maybe there is some reason that is not allowed.

[1] [http://www.brownrudnick.com/nr/pdf/alerts/Brown%20Rudnick%20...](http://www.brownrudnick.com/nr/pdf/alerts/Brown%20Rudnick%20Advisory%20The%20Bionic%20Library- Did%20Google%20Work%20Around%20The%20GPL.pdf)

<comment><USER4> I'm sure that Linus does not think that programs that run on the Linux kernel should have to be GPL'd. I doubt if Angry Birds or Flash use kernel header files -- they call the kernel, yes, but as userspace programs not as part of the kernel.

<comment><AUTHOR> Linus Torvalds says that "normal system calls" are allowed, but once you use the original kernel headers, you're subject to the GPL: <http://lkml.org/lkml/2003/12/5/13> The way to avoid this is to use glibc.

<comment><USER5> In relation to Android and Bionic, I think the following message from the same thread is more relevant:

<http://lkml.org/lkml/2003/12/5/162>

(in general, the thread was about kernel modules - this message explicitly mentions user-space)

</comment></comment></comment></comment></comment><comment><USER6> Oy, this guy _again_? For those unaware, Florian Mueller has made a mini- career of spreading anti-FOSS and anti-Android propaganda.

      > Google copied 2.5 megabytes of code from more than 700
      > Linux kernel header files with a homemade program that
      > drops source code comments and some other elements, and
      > daringly claims (in a notice at the start of each
      > generated file) that the extracted material constitutes
      > "no copyrightable information".

In the US, APIs are typically considered mere specifications, and cannot be copyrighted. You can copyright a particular encoding of an API (eg, a C .h file), but the API _itself_ can be used by anyone. If APIs could be copyrighted, projects like Wine and Mono would be illegal.

Whether Google's libraries are considered derived works will depend on how the tool works. If it just regexes out all the comments, then the result is probably a derivative work. If it parses the headers and then outputs the API information, the output is probably not a derived work.

      > The GPL's copyleft nature requires all derivative works
      > of a GPL'd program to be made available on the same
      > terms.

Wrong; the GPL requires any derivative works to be under a _compatible_ license. If I copy GPL-licensed code into my BSD-licensed library, none of my code has to be relicensed.

      > If Google is proven wrong, pretty much that entire
      > software stack -- and also many popular third-party
      > closed-source components such as the Angry Birds game
      > and the Adobe Flash Player -- would actually have to be
      > published under the GPL.

Completely and egregiously wrong. Unless these programs are actually copying parts of the kernel into their source, they are not derived works.

Come on, this is software copyright 101. Copyright doesn't propagate from a library to its clients; otherwise, every Windows application is infringing Microsoft's copyrights, every iOS app is infringing Apple's, etc.

      > The only realistic way to fix the problem: replace
      > Bionic with glibc

How would this help? According to the author's reasoning, glibc is also an illegal derivative work of the Linux kernel. Any application which infringes if it uses Bionic will also infringe if linked against glibc.

The rest of the post is a bunch of doomsday scenarios, all built on the incorrect assumption that running proprietary software on a Free kernel/libc is illegal.

<comment><USER7> _If I copy GPL-licensed code into my BSD-licensed library, none of my code has to be relicensed._

Yes it does, that's a pretty central feature of the GPL. See section 2.b. of the GPL (v2):

 _b) You must cause any work that you distribute or publish, that in whole or in part contains or is derived from the Program or any part thereof, to be licensed as a whole at no charge to all third parties under the terms of this License._

<comment><USER6> You forgot the very next paragraph:

 _These requirements apply to the modified work as a whole. If identifiable sections of that work are not derived from the Program, and can be reasonably considered independent and separate works in themselves, then this License, and its terms, do not apply to those sections when you distribute them as separate works. But when you distribute the same sections as part of a whole which is a work based on the Program, the distribution of the whole must be on the terms of this License, whose permissions for other licensees extend to the entire whole, and thus to each and every part regardless of who wrote it._

In other words, my code remains mine. I still control the copyright to it, and do not have to relicense it.

<comment><USER7> Those conditions don't apply because Google is accused of distributing a derived work. If Google is distributing a work derived from GPL'd code then they can only distribute that work under the terms of the GPL. There's a lot of debate over whether what Google's distributing can actually be considered a derived work, but that's a separate question entirely.

</comment></comment></comment></comment><comment><USER8> Odd that he doesn't explicitly mention the legal basis for Google's claim that this stuff is non-copyrightable. Which is this:

Copyright protects the expression of an idea, not the idea itself. So, the courts have held, if you don't have any choice over how to express something, it isn't subject to copyright protection. In particular, if a particular mapping from names to magic numbers is necessary in order for standard- conformant programs to operate at all (as in the mappings specified by <errno.h> and <signal.h>, which came up in the SCO litigation), that stuff isn't copyrightable.

So, Google's saying, they've written tooling to explicitly drop all the expressive elements, leaving the bare, functional, non-copyrightable bones --- which they don't need legal permission to use, from Linus or anyone else.

This is, I guess, why Mueller thinks it's a big deal that the processed files still include inline functions --- if there's another way to write the function which does the same thing, then there is an expressive choice in writing it, which would be subject to copyright protection.

But if userland code has no reason to want to run particular inline functions in kernel header files, then Google has an easy remedy: just ditch 'em --- or rewrite the few that matter, and ditch the rest.

So, this really doesn't look like a big deal...

<comment><USER9> I haven't looked at the code in question, but there are some inlined functions. If there aren't too many and the functions are very short, then it's probably fine.

Just copying the #defines and stuff is almost certainly okay since you would need to duplicate all of that to call into the kernel.

There is precedent for something like this: Phone books have been copied, proved to have been copied (there were fake entries in the original) and it was determined not a copyright violation.

</comment></comment><comment><USER10> The Linux kernel _specifically_ allows programs to make system calls into the kernel without being GPL. From <http://lxr.linux.no/linux+v2.6.38/COPYING>

        NOTE! This copyright does *not* cover user programs that use kernel
        services by normal system calls - this is merely considered normal use
        of the kernel, and does *not* fall under the heading of "derived work".
        Also note that the GPL below is copyrighted by the Free Software
        Foundation, but the instance of code that it refers to (the Linux
        kernel) is copyrighted by me and others who actually wrote it.

The Linus quote on the other hand, pertains to _binary modules_ \-- ie. binaries that are linked into kernel-space and use _non-system-call_ APIs.

Since user programs are specifically exempt from being GPL'd, I cannot see how you could possibly make the argument that Angry Birds is somehow affected, unless it is being linked in as a kernel module.

What is in these headers? If it is just definitions that are required for making system calls, I would expect that all of this falls under the exception above.

</comment><comment><USER11> I think the part of the article everyone should make sure to focus on is that Google is doing exactly what Linus says is impermissible. "In short: you do _NOT_ have the right to use a kernel header file (or any other part of the kernel sources), unless that use results in a GPL'd program." "So you can run the kernel and create non-GPL'd programs [...] BUT YOU CAN NOT USE THE KERNEL HEADER FILES TO CREATE NON-GPL'D BINARIES. Comprende?" - Linus Torvalds quote, taken from the linked article

<comment><USER12> An excellent point, though (unfortunately) just because Linus wants the GPL to apply that way doesn't necessarily mean that it does.

As a layman, it seems entirely reasonable that you should be able to inspect the interface of a software product in order to make something compatible with it without your new product counting as a "derived work." If they had read documentation for all the functions rather than parsing the headers through a script would that have made a difference?

Put another way, I'm sure Microsoft asserts that it is impermissible to reverse engineer Windows kernel code, but that doesn't mean they have the power to stop it.

</comment></comment><comment><USER13> As I understand it, Google is using the header files only to extract "structure declarations, constants, etc." -- i.e. to get the interface right, not to link to kernel source. The question is whether or not this is a copyright violation... If not, then the GPL has no power.

Note that it's possible to use a copyrighted source to reverse-engineer an interface and not have it be a copyright violation: <http://en.wikipedia.org/wiki/Clean_room_design#Case_law>

I'm hoping this is a lot of fuss over nothing, as there will no doubt be cases in the future where people want to create open-source drop-in replacements for proprietary systems, and if it is ruled that extracting information from header files makes a derivative work, it will be very difficult for them to get the interface right.

</comment><comment><USER14> It's hard to take the author's claims of infringement seriously because of his laughable claim that third party applications such as Angry Birds running on top of Android would have to be open sourced under GPL if actual infringement is found in Android.

<comment><AUTHOR> The reason is that Angry Birds is written in C and certainly accesses the Linux kernel, almost certainly using the Bionic library. If that one is GPL'd, Angry Birds has to be GPL'd. GPL copyleft affects derivative works, and compiling a library into your program is a clear case of a derivative work.

<comment><USER15> No, no, no. That's wrong. It's the "viral" misinterpretation of the GPL at work. Angry Birds code belongs to Angry Birds' developers, period. They get to decide its license, period. There are _no_ circumstances under which "Angry Birds has to be GPL'd". None.

The issue at hand is whether or not the Angry Birds Android port constitutes a derived work of the kernel (a pretty strained case, IMHO, given that it's a straightforward port of a game from another non-linux platform). If it is, then it can only be distributed under the GPL -- that is, it'd have to be pulled from the market. At no point does it get removed from its owner's control.

<comment><USER16> >The issue at hand is whether or not the Angry Birds Android port constitutes a derived work of the kernel (a pretty strained case, IMHO, given that it's a straightforward port of a game from another non-linux platform). If it is, then it can only be distributed under the GPL -- that is, it'd have to be pulled from the market. At no point does it get removed from its owner's control.

The FSF states that if any program links against GPL licensed libraries, it is a derivative work, and therefore needs to be under GPL as well. The LGPL was created specifically to allow linking by code from other licenses.

While the Angry Birds code still belongs to the Angry Birds developers. However, if Google is guilty of a GPL violation, and are unable to get a LGPL licensed wrapper around the code, Angry Birds for Android may need to be distributed under the GPL to avoid a violation.

Most likely, Google will find a way to make an LGPL wrapper, and this entire issue won't matter.

<comment><USER17> Rovio will still never be forced to distribute Angry Birds under the GPL, even if they are found to be a derived work by association. Being found of infringement just means they have to stop distributing Angry Birds, _in it's infringing form_. Once the infringement is fixed, Rovio can then distribute the non-infringing form of Angry Birds without having to license it as GPL.

If however, Rovio wants to continue distributing the _infringing_ form of Angry Birds, they only license they are legally allowed to use for that is the GPL.

But at no point is Rovio ever _forced_ to distribute a GPL licensed version of Angry Birds.

<comment><USER16> It is true that Rovio isn't forced by anyone to create a GPL version or to release their code. I should have pointed this out.

However, their options are pretty limited, considering they want to make money off of the software. They could simply stop distributing it from the platform. But they want to make money, which means that they(presumably) want to get as many sales of the game as possible on as many platforms as possible.

However, if the main system interface is GPL(and not LGPL), the GPL does not allow anything to link against it that is not GPL code. So, given that, they are effectively forced to do it if they want to get any further sales and ad revenue on the Android platform, because they would not be able to distribute the game in it's current version.

</comment></comment></comment></comment></comment></comment><comment><USER18> How not to get snookered by claims of "proof" of copyright infringement: <http://www.groklaw.net/article.php?story=20110122054409107>

<comment><AUTHOR> That Groklaw article failed to explain that the code in question is indeed distributed online by major Android device makers such as Motorola, Samsung and LG: [http://fosspatents.blogspot.com/2011/01/android-device- maker...](http://fosspatents.blogspot.com/2011/01/android-device-makers- distribute-oracle.html)

<comment><USER18> I'm not claiming it as truth - just letting people know that there's a history here and opinions on both sides.

<comment><AUTHOR> Thanks for clarifying that. Yes, let freedom and pluralism ring.

</comment></comment></comment></comment><comment><USER19> I think the issue is whether or not header files are copyrightable. In verbatim form, they are, but it's easy to recreate identical header files without copying the source code: have someone read the header file, write a list of functions and signatures, use this spec to write an identical header. Google wrote a program to do this.

I am fine with the outcome either way. If everything on Linux has to be GPL'd, excellent: more freedom for everyone. If Google's stuff is OK: that makes sense too, because the OS's copyright shouldn't infect its apps'... apps have to use an OS.

I love not having a pony in the race.

<comment><USER20> only problem the SCO case proved them not copyright-able..

whoops..

</comment></comment><comment><USER21> Oh my, oh my, oh my...

 _Developers of applications using Bionic would only be able to charge (via the Android Market) those customers who don't know what rights they have under the GPL. All others would find ways to download and install those apps on GPL terms, i.e., free of charge._

Mr. Mueller doesn't seem to understand that if something is released under GPL that _does not_ make it automatically free of charge. Just because most GPL software is free of charge, that does not mean that you can't charge for your software if you release it as GPL.

Another common misconception is that if your code is under GPL, you have to give the source code to just about anybody who wants to see it for free. No. Wrong. Only people who have the license for your software can get the source code, they have this right because it is granted to them by the GPL. And if you want to, you can certainly charge for use of this license.

So there is nothing stopping you from releasing your software as GPL, charging for it, and giving the source code to those interested parties that payed for the license to use your software.

<comment><AUTHOR> @Sandman, the "oh my, oh my, oh my" part is actually justified as a reaction to some of what you say.

Of course you can try to charge for GPL'd software. But since the GPL requires you to make it available free of charge, there would be arbitrage: people would find where to get it for free, and why would they pay then?

<comment><USER22> The GPL does _not_ require you to make your software available free of charge. It merely states that if someone purchases software from you, that you must also provide the source code upon request.

<comment><USER23> But it requires that those who receive licenses from you (or source) be able to pass them to others, free or charge or not.

<comment><USER24> So what?

It assumes that whomever wants to get your $3 program can:

* Know exactly what the rules are under the GPL (which as we have just seen, even smart software developers can't).

* Knows somebody who already have access to the software so he can get the source code.

* Trust the person whom he gets the software from.

* Knows how to compiler source code (or his friend has to), which means there has to be at least one programmer at some point. Programmers (especially with mobile programming experience) are not cheap.

* Is able to install the software, which again requires above average computer skills (many people don't know how to activate the menu)

Edit: formatting

* You will most likely be stuck with an out-of-date version, since Android programs are updated fairly often

So unless you sell something where the majority are computer programmers, or are selling for a very high price, you properly wouldn't be able to get it for free.

Edit:formatting.

</comment></comment></comment></comment></comment><comment><USER25> A couple quick points about copyright and header files.

1\. It is well established in US copyright law that elements necessary to interoperate with something aren't copyrightable. So, for instance, the function signatures and probably function names (if names are necessary in order to make the call) of an interface would not be copyrightable. Same for structures used in the interface.

This has been fairly extensively litigated. It arises when you have, for instance, someone trying to make an unauthorized third party game for a video game system, or someone trying to write a plug-in for someone else's closed program.

2\. Header files often contain more than just the bare elements necessary to interface to something. They often contain macros that expand to code, comments, maybe even implementations of small inline functions. There is plenty in header files that can be copyrighted.

So, depending exactly on how Google produced its header files, they may be in the clear or they may have made a big mistake.

<comment><USER26> According to Naughton's analysis [1], your case 2 applies, and there's also the consideration that the whole constitutes an expression, and is therefore copyrightable.

It's worth reading the original instead of the above linkbait.

[1] [http://www.brownrudnick.com/nr/pdf/alerts/Brown%20Rudnick%20...](http://www.brownrudnick.com/nr/pdf/alerts/Brown%20Rudnick%20Advisory%20The%20Bionic%20Library- Did%20Google%20Work%20Around%20The%20GPL.pdf)

</comment></comment><comment><USER27> So wait, the grumble is that they extracted function signatures from a header file? What a non-event.

</comment><comment><USER28> I don't get it. The LGPL lets you promote to the GPL, but the GPL does not give any special status to the LGPL; it doesn't even mention it outside of the preamble. Why is going through a LGPL library any different than going through a BSD one?

</comment><comment><USER25> I am at a complete loss as to why no one like Google, or a consortium of smaller embedded system developers or users, have not made a BSD based system available as an alternative to Linux based systems.

Linux has some desktop advantages in terms of package management and perhaps consumer hardware support, but none of these matter to someone seeking an OS for a phone or a TV or a microwave oven and so on.

<comment><USER29> The precursor to Android was actually BSD-based instead of Linux-based. I assume that when the Danger guys left to start over with Android, they figured starting from Linux would give them a technical advantage.

</comment></comment><comment><USER30> This may or may not be true, but what is fundamentally different from the Oracle suit is that the GPL holders take a very pragmatic view of violations. Release the code as GPL, or stop using it, and you're ok. I never heard of a GPL suit involving damages... not even sure it's possible.

Oracle on the other hand wants either to hurt Android, or a (preferably large) piece of the pie.

<comment><USER31> "The SFC received a default judgment [PDF] against Westinghouse in July for $90,000 in damages and $50,000 to cover costs of the suit."

<https://lwn.net/Articles/409303/>

</comment></comment><comment><USER32> Interesting... Assuming the article is correct in that Google's tool says that no copyrighted information is left, it seems that Google is saying that header files stripped of all of english/whatever-language text and left with only pure code are not copyright-able because it's just code?

<comment><USER33> The argument is that it's just _interface_, which has to be that way for interoperability, so has no "creative content". Code that is actually expressive code in a header file (inline functions, or non-constant macros) could readily spoil this. I have no idea whether the output headers contains more than structure definitions, constant macros, and forward function declarations.

</comment></comment><comment><USER34> This is what you get when lawyers try to understand developer stuff. Plus, it sounds like more FUD to me.

<comment><USER3> Or possibly it's what happens when developers who don't understand legal stuff try to circumvent software licences.

</comment></comment><comment><USER35> would it be possible to relicense the output of google's tools (the rewritten kernel header files) to LGPL with the permission of the kernel developers (just like glibc or uclibc do) and build a separate library out of this? This library would consist mainly of header files. And this library can then be linked against the rest of bionic whereat bionic could keep its current license. Would this be ok?</comment>