Are unsound type systems wrong? - <AUTHOR> http://frenchy64.github.io/2018/04/07/unsoundness-in-untyped-types.html <comment><USER4> It would be nice to have some examples of what these terms mean, for type system dilletantes like me. I must say, I was a little lost :)

Re: TypeScript:

 _> TypeScript challenges the status quo, claiming that lowering the cognitive overhead of using types is more important than type soundness. They argue that JavaScript programmers, their main audience, would find it easier to use TypeScript if they deviated from traditional norms._

Im not sure about the official mission statement, but as far as I understand, the purpose of their typesystem is to allow static description of the way existing JavaScript is typed, _de facto_ , in the wild. This explains why e.g. literal strings are a type so you can do:

        interface Eventor {
            on('foo', FooHandler);
            on('bar', BarHandler);
        }

Not because it's nice to have a type that "foo" satisfies and not "bar", but because there is actual code out there that does this, today. Same for structural typing (TypeScript would be absolute horror to interface with existing code without it). And, I assume, covariant arrays.

TypeScript was not created in a void, as a fresh, new typesystem. It was created to formalise the way existing JavaScript was informally typed. It would have been nice to see a little reference to or acknowledgement of this.

Or an example of how this could have been achieved better; preserving type soundness.

<comment><USER5> There are basically two important results for type systems, soundndess and completeness.

Soundness means that every program that typechecks is valid. Completeness means that every valid program can be typed.

So basically typescripts type system doesn't match the language semantics so type checked programs may crash at runtime.

<comment><USER6> I'm curious, given Godels incompleteness theory, isn't it impossible to have both soundness and completeness in the same language?

<comment><USER7> Gödel's incompleteness is for proof systems for languages which contain enough arithmetic to express its own syntax. Lots of things don't go this far - <USER2> order predicate logic is sound and complete, for example (see Gödel's completeness theorem(!)).

<comment><USER8> Could you please expand on the soundness and completeness of first order logic? Isn't that false generally? Because you need a deductive/axiom system, so that you can ask the question of completeness/soundness. And ZFC is too strong, hence cannot prove its soundness and correctness at the same time. But you can formalize weaker systems in first order logic, that are provably complete and sound.

<comment><USER9> (It's been a year and a half since I took a course on this, so I apologize for any mistakes)

You do need to define a deductive/axiom system before you can ask the question. We can use a few different standard deductive systems. We require that the axioms are sound. Then the system is complete. (I think one should also always be able to prove false if the axioms aren't sound... but I don't recall proving that)

ZFC isn't too strong. That sounds like a contradiction, but it isn't, because we don't have a rich enough vocabularly in first order logic to state the problematic statements that make it either inconsistent or incomplete in stronger logic systems. Every true statement you can state about ZFC _in first order logic_ is provable.

<comment><USER10> > It's been a year and a half since I took a course on this

Can you please recommend a MOOC/resource for learning more about this?

<comment><USER9> My only real exposure to it is the course I took, csc438 at UofT taught by Steven Cook.

There are pretty complete notes for the course as well as assignments here [0], but not videos or planned lessons. You certainly could learn about it by reading them, but I don't know if it would be the most efficient way.

[0] [http://www.cs.toronto.edu/~sacook/csc438h/](http://www.cs.toronto.edu/~sacook/csc438h/)

</comment></comment></comment></comment></comment></comment></comment></comment><comment><USER11> I've worked a lot with TypeScript after a background in more type-y languages like O'Caml and Haskell and have eventually gained a real appreciation for the engineering tradeoff they made. This article sorta frames soundness like it's a feature -- one could have soundness or not, so why not have it -- when in really a specific technical term about a property of a language that programmers often don't understand (as evidenced by the other comments on here) and which has specific consequences.

Instead, I now view type systems as a tool that can make tradeoffs to get the job done, in the same space as "IDE" or "test-driven design". With those it's easy to make statements about the specific programmer tradeoffs, such as "test-driven design means you spend more time writing tests but gives you more confidence about the results". With a given type system, you can evaluate it like "type system X lets you get autocomplete in your editor", or "type system Y catches the common error of Z".

Meanwhile if you only focus on soundness you end up with counterintuitive results like where Java is sound but still has NullPointerExceptions everywhere, or languages like Rust where whole data structures and algorithms are disallowed because you can't express their correctness to the compiler. (To be clear I appreciate exactly why Rust must do this, but again you can evaluate that tradeoff as "preserves memory safety without GC, disallows doubly-linked lists" rather than "is the type system 'correct'".)

One view that really opened my eyes on this Gilad Bracha's talk on pluggable and optional type systems: [http://lambda-the- ultimate.org/node/1311](http://lambda-the-ultimate.org/node/1311) .

<comment><USER12> > whole data structures and algorithms are disallowed

Nothing is disallowed in rust. You may need a specialized crate or a small 'unsafe' block to work beyond the compiler's ability to prove your code meets rust's guarantees, but that's not the same as inexpressible.

<comment><USER13> Unsafe rust is a superset of safe Rust. They're two different languages. The OP is saying, correctly, that certain data structures are inexpressible in safe Rust. That's fine. Certain control flow structures are inexpressible in goto-less C.

<comment><USER14> > Certain control flow structures are inexpressible in goto-less C.

I'm not sure about that. C contains while loops. And every GOTO-computable programs should be expressible by WHILE-computable programs.

See (slide 21): [http://ai.cs.unibas.ch/_files/teaching/fs16/theo/slides/theo...](http://ai.cs.unibas.ch/_files/teaching/fs16/theo/slides/theory-d03.pdf)

<comment><USER15> I don't think "certain control flow structures are inexpressible" is the same assertion as "certain programs are unexpressible"...

After all, any turing complete system will have a similar set of computable programs as another, but they will not all have the same control structures. For example, you can have a turing complete language with no loops, only recursion. Both can express the same programs but not in the same way.

That said, it's also a bit of a tautology so.

</comment></comment></comment></comment></comment><comment><USER16> Unsoundness is important because soundness and expressiveness come at a trade- off.

Here's some code which is syntactically legal in both TypeScript and Flow, but the languages disagree on whether a type error is present:

      type T = { // Legal decl in Flow, illegal in TS
        x: number;
        [s: string]: boolean;
      };
      const m: T = { x: 1, xz: true };
      m['xz'.substr(0, 1)] = true;
      m.x.toFixed(); // Crashes

TypeScript is actually _more_ sound than Flow in this example because it disallows the declaration of T in the first place. Flow allows this code and it fails due to a type error at runtime.

Yet in JavaScript this is an extremely common pattern - an object bag where some known property keys have a certain type, but "all the others" have a different type. You can't represent this in a bolt-on type system without forcing the developer to jump through enormous hoops proving that any lookup key _isn 't_ one of the "known" keys. By enforcing soundness at all costs, you lose the expressiveness needed to ergonomically describe this object.

Since most people don't directly aim guns at their feet, the unsoundness here is unimportant in practice. This is one place where I think Flow made the right trade-off to be unsound for the sake of practical usability for a common pattern.

<comment><USER13> Why wouldn't you just make your untyped bag a field of the larger and completely specified object then?

<comment><USER16> "Will it be easy for a computer to prove that my API is being called correctly?" is not a question that JavaScript library authors seem to give any thought whatsoever to

</comment></comment></comment><comment><USER17> To be honest, after reading this article, I don't really have more information why you'd want to use an unsound type system at all.

If you have a sound type system, it's harder to write code but the compiler and IDE can infer useful invariants and give you guidance in return.

If you have no type system, you're on your own with error detection, but writing code may be a lot faster.

(Though I personally think a modern language with sound types and useful type inference can solve a lot of the tedium as well)

If you have an unsound type system you get the worst of both worlds: You have the cognitive overhead of a typed language but at the same time don't get any of the guarantees and invariants.

<comment><USER18> A big advantage of unsound types is versioning. IE you can add some member variables/methods to a structure/class and users of that library that were written with the old version still work without trouble, recompilation or redeploying. Of course if you remove the variable they're in trouble but removing can be avoided for that reason.

<comment><USER19> In this sense typing is like unit-testing it would seem. You can make your program "better" by adding more unit-tests to it. But more tests you have the more work it will be to modify your tests, to accommodate the changes in your program.

The more strict your type-system is the more things you can prove and thus understand about your program. But if you need to change your program it may no longer compile with the existing types. Does your type-system then make it easy to change your types so they reflect your new program?

Now assumably when you are developing a new program it is changing all the time. If you fix its types early doesn't it actually make it more work to create your program?

Once your application is done and works and is big it is essential that it is "typed" so that you can see what changes don't require changes in the types vs. what changes would require a lot of rewriting of the types all over the place.

I think the fact that many useful programs are written in dynamically typed languages speaks for this. Small programs don't need types.

Does this mean that optional typing is a good feature?

<comment><USER8> I usually find a lot more serious bugs in quick and dirty scripts, than in big systems, regardless of language. (Naturally, big systems have already survived the small PoC phase, plus usually big things are viewed as complex, whereas small scripts are dangerously innocent looking.) So in that way things like Ammonite (easy scripting using Scala) seem very useful - even if the runtime requirements are a lot heavier, than using bash (or python).

Furthermore, usually development uses type system as a tool, so it'd be hard to add typing when it's done. (I mean, usually functional doneness is checked or "asserted" in a large part through types, and unit/integration tests help with the rest.) So in this sense typing doesn't add more development time. It just makes problems a lot more explicit, so you know you're not done yet.

<comment><USER19> Explicitly declared types are like anchors. They make it harder to change the program to be a different program. In many cases this is good since it tells whether a change breaks things or not. But it does add to the "mass" of your program, makes it harder to change its direction.

Your declared types are the shared "language" under your program. It is much harder to change the language that what is said in it. Changing the language would actually change the MEANING of what has already been said.

</comment></comment></comment></comment></comment><comment><USER20> Yes: your program fails at runtime, and you have no idea of where the problem is, because the place where the type invariants are violated is not related to where an exception is thrown or an incorrect result is produced.

It's basically like using and debugging C/C++ code, where it looks like there is a "type system" but in fact you are just arbitrarily manipulating a big byte array that contains all your data and the call stack (except with TypeScript it's a graph of untyped arrays and dictionaries instead).

<comment><USER21> It is absolutely no different from debugging Javascript. In Javascript you have an expectation of types in your mind, that are not guaranteed. The same is true of Typescript. The difference is, Typescript will catch most exceptions to the type system at compile time. It is not sound, but is extremely _useful_.

<comment><USER22> "Most" isn't necessarily a good thing, because it can lead to a false sense of security. For example, if you start thinking, "no need to check for null here, because the types won't allow it", you're open to being surprised when a null sneaks in at run-time. So you have to deal with the overhead of both types and run-time guards.

<comment><USER21> This is the sort of thought people have when they haven't actually used the language much. It is also an example of something Typescript is really good at. If you try to use an `any` value - or any nullable typed value - <USER3> hasn't been checked for null, it is a type error. Sure you could write a type definition for a Javascript function that is a lie and say that it cannot return a null when it can, and then guess what? You'll get a runtime error and debugging it is identical to Javascript.

In fact I do not know of any language that is actually safe at it's FFI boundaries. Rust isn't, Haskell isn't. You could get a value back from a C function that you were told would be allocated 64-bytes and actually it was only 32, so you'll have an overflow somewhere later. The Typescript / Javascript boundary is similar, though obviously not as dangerous.

<comment><USER22> The problem is in just how much boundary there tends to be, since many TypeScript programs are heavily mixed with ordinary JS. I would agree with you for programs that exclusively use TS for application code.

</comment></comment></comment></comment></comment><comment><USER23> It's an interesting question. It seems that developers are quite able to cope with weird type systems, moreover most of them don't even think about it or even care.

A simple example of python2 that would make developers brought up on a healthy diet of haskell cringe is this:

       >>> for x in range(100): print type(1<<x)

And yet, pythonistas seem to mind nor bother.

<comment><USER24> This kind of thing is why I'd favour two different types of numeric types:

\- "Number", which would be the generic DWIM arithmetic type. In your example both x and the literals would be Number.

\- Hyper-specified arithmetic types. Not just number of bits and signedness, but also overflow and trap behaviour. Less convenient to use - you can't easily add a "32bit signed wrap on overflow" to "16 bit unsigned saturate on overflow", but have the advantage that there is no unspecified behaviour. This also allows the user to make use of machine-provided saturate and overflow behaviour where present.

<comment><USER25> I see where you are coming from, but C offers a cautionary tale: It offers more choices than most languages, and that creates a number of traps for the unwary. Maybe, however, that is only because it allows mixed-type expressions?

<comment><USER24> Choice for the programmer, or choice for the implementer? Part of the problem of C is the amount of stuff with was left as a choice for the implementers on a particular platform and therefore could not be safely assumed to be universal. Forcing the programmer to enumerate what they actually want from their "fast machine integers" avoids that.

C offers a fairly small choice of types to the programmer, certainly compared to one of the ML/Haskell style of language. It doesn't even have a proper "string" type!

C doesn't exactly allow mixed-type expressions, it just performs a lot of type coercion and reinterpretation. Which is another one of those things that trades safety for convenience in so many languages.

Type flexibility is responsible for most of the wtf type moments in Javascript: [https://www.destroyallsoftware.com/talks/wat](https://www.destroyallsoftware.com/talks/wat)

</comment></comment></comment></comment><comment><USER26> Unsound typesystems are not necessarily a problem, but consider the problem the other way around: given a sound typesystem, how do we improve this in such a way that a) the average Joe will understand it b) it decreases the overhead of writing the types c) the type system helps the programmer write better programs

Haskells approach to this keeps the constraint of soundness: [https://youtu.be/re96UgMk6GQ?t=52m5s](https://youtu.be/re96UgMk6GQ?t=52m5s)

The whole video is very interesting even if you know nothing about Haskell, but are just interested in language design in general.

To summarize the part of the video I just linked: Their approach so far has been to try and design the type system in such a way that you reduce programmer pain while not sacrificing soundness at all. It is a painful constraint for language designers, but it allows you to gradually add functionality to the type system from observing real world examples. The goal can be described as: Design a typesystem such that every working program can be typed in both a sound and correct way, while minimizing the amount of programs that can be typed, but are not working as intended.

</comment><comment><USER27> I don't really think that unsound type systems are wrong, because I think it depends on the role of the types in the programming languages. Typescript, Java etc. don't have that sophisticated type systems and you can just check the types for yourself very quickly. You rely on the type-system to correct obvious mistakes, but not more.

In Languages like Haskell or completely dependently typed languages it would be morally wrong (in my opinion) to have an unsound type-system. Here the role of the types is different, you use types to compute types and need to trust the compiler in complicated situations that this is sound, just like you trust the compiler to do correct floating arithmetic. Checking the types per hand may be a real challenge, for example if your writing super polymorphic code. Sometimes you even have to prove that your code fits the type-signature (per induction etc.), here you want guarantees.

</comment><comment><USER28> > The way TypeScript was designed is in fact more promising than older systems > like Java. Those systems inadvertently broke type soundness instead of > making it a deliberate action.

Not sure what the article is referring to here...

Java soundness issues like array covariance, final field mutability, unchecked generics casts, etc were definitely surprises to many users, but the Java language designers were well aware. They added runtime checking to make sure those soundness holes didn't cause the JVM to crash or violate the security guarantees.

Sure, there were some additional surprises, but I'm sure TypeScript has had many more. (Partly because the TypeScript type system is more complicated, but also because it's not important to get it 100% right -- JavaScript runtimes don't perform optimizations that rely on the soundness of TypeScript.)

</comment><comment><USER29> _Wrong?_ That's a bit of an odd question. It's only _wrong_ if the context is, "what is an example of a sound type system?"

Now, there are distinct disadvantages to unsound type systems, but that doesn't make them "wrong". I love Python and sometimes wish I could turn on a switch in the interpreter that made it statically typed, but there is no denying there is a freedom and (initial) productivity boost in ducktyping.

<comment><USER30> Python does have static typing nowadays :) Checkout Mypy, and also Dropbox's PyAnnotate, which helps statically type existing sources by recording types during runtime and then adding them to the code.

</comment></comment><comment><USER31> Dunno about "wrong" but I really do like having what the author calls "open- world" soundness in Typed Racket.

Other than that it really depends where. How bad unsoundness is depends on how sneaky the resulting errors are and how hard they are to debug.

</comment><comment><USER32> I think some of the recent developments in run-time type checking systems like the one in Mobx-state-tree [1] and Clojures Spec [2] are interesting. I don't know enough about either to make any witty remarks but I'd like to see the impact they have on the field.

[1] [https://github.com/mobxjs/mobx-state- tree#thanks](https://github.com/mobxjs/mobx-state-tree#thanks) see tcomb

[2] [https://clojure.org/guides/spec](https://clojure.org/guides/spec)

</comment><comment><USER33> Most readable introduction to the topic I've read so far. Nicely outlines the dichotomy between unit testing and types as both serving as different kinds of verification tools.

Types as "approximation of the result of a computation" is also a nice and concise way to describe them, haven't seen that definition before.

</comment><comment><USER34> TypeScript cannot be sound because its model of the runtime environment necessarily diverges from the runtime truth. A simple example: some properties were removed in ES6, so type checking against an ES5 runtime will result in type errors when run in an ES6 browser, and vice versa.

Soundness only provides a guarantee if you control when and how your code will be executed, and that's a tough constraint. To prove is not to forsee the future.

<comment><USER35> > some properties were removed in ES6

Which properties are you referring to? My understanding is that ES6 is back compatible and so that can't be case. There's "use strict" but that's a different kettle of fish; a pragma that changes runtime behaviour..

</comment></comment><comment><USER36> People arguing that type systems like TypeScript are useful, because even if they don't catch all errors, they still catch the most common ones.

The most common JavaScript error I encounter is accessing fields of undefined objects.

Can't we simply have a type-system for JavaScript that only prohibits undefined field access and nothing else?

If we accept that the system will be unsound anyway, why not focus on the main benefit and throw all the baggage out?

<comment><USER37> > The most common JavaScript error I encounter is accessing fields of > undefined objects.

TypeScript has optional null checking if you want it. Given that it supports union types, null/undefined tracking fit nicely into the language.

<comment><USER36> Can I type stuff as any but still non nullable?

<comment><USER37> I think any in TypeScript's context means "not in the type system", and all type rules bounce off it. So if k's type is any, and you type k.foo, it doesn't bother checking it. I assume the same applies to any null checking.

<comment><USER36> The "object" type is probably what I was searching.

<comment><USER38> The object type is the type of all non-primitive types, what you want is the Object type instead. Yes, it's confusing, especially as there's also the {} type! This article goes over the differences:

[https://blog.mariusschulz.com/2017/02/24/typescript-2-2-the-...](https://blog.mariusschulz.com/2017/02/24/typescript-2-2-the- object-type)

</comment></comment></comment></comment></comment></comment><comment><USER39> I feel like a distinction needs to be made between soundness and precision. A sound type system never tells you something wrong while a precise type system tells you everything that needs to be said about a value. Gradual type systems are imprecise, because they could type something at some * type. There's nothing wrong with that. Unsoundness is bad on the other hand, it keeps you from omitting dynamic type checks in places where you would want to in your compiler. A natural thing to want to do is include dynamic checks expressly where you are downcasting from a * type, omitting them where the type is known precisely enough. That is how you reap a performance gain from your type system, and unsoundness prevents you doing that.

(Downcasting from * in a gradual type system is analogous to how C lets you cast from void* to any pointer type implicitly, except that RTTI checks that the cast is safe.)

</comment><comment><USER40> Flow has a good page on this subject: [https://flow.org/en/docs/lang/](https://flow.org/en/docs/lang/)

</comment><comment><USER41> > Programmers often use other verification tools like unit testing or > contracts to get strong assurances of similar properties.

Neither unit testing nor contracts are verification tools.

</comment><comment><USER42> Is typescript wrong? Maybe, but if it is wrong, I don't want to be right.

After switching all my JavaScript Dev to ts a couple years ago, I cant imagine how I would get by without it.

</comment><comment><USER43> I tend to call Typescript's type system "pragmatic typing":

\- type inference

\- structural types

\- union and intersection types

\- _important_ : easy escape hatches such as trivial cast to `any` or the ability to write your own type definitions for almost any random piece of code including third party libs

Type purists may scoff at Typescript's type system, but it's there, it mostly works, it's very easy to get started with, it's reasonably fast. I would personally wish for exhaustive type checking, and I wouldn't be surprised if I get my wish sometime in the future.

Another rather pragmatically typed language is OCaml/ReasonML

</comment><comment><USER44> That's an unusual use for "type soundness". To me, type soundness has more to do with whether all the functions in a program are total, i.e. defined across their entire range of inputs (can't throw exceptions, can't crash, always return values belong to the return type they declare).

<comment><USER45> The author's use of "type soundness" matches the precise, widely-accepted mathematical definition which is used in the literature. Type soundness does not require total functions, although they certainly make it simpler.

It is possible to encode the semantics of exceptions such that a reasonable definition of soundness is still provable. See, for instance, Chapter 14 of Types and Programming Languages by Pierce.

</comment></comment><comment><USER46> What is a high level type system anyway ? Annotations to help feed the intellisense engine, to warn about miss-spelled variables, make sure you don't use an apple where there should be a banana, Or are types just objects with limited states !?

</comment><comment><USER47> I miss MeeGo

</comment><comment><USER48> > Interestingly, fixing these areas in TypeScript has been a fruitful source > of new research

That's not interesting. TypeScript did things in way Y, different from the usual way X, and some academics realize they get a free paper out of reimagining TypeScript doing X.

</comment><comment><USER49> Many things in the computer science fields are wrong. If people want to have blockchain and turn one security problem into six, I won't stop them.

Let us synchronize ALL of the computers, while we're at it.

Also, suddenly cryptography and computers are cool :)</comment>


YouTube Identifies Birdsong As Copyrighted Music - <AUTHOR> http://c4sif.org/2012/02/youtube-identifies-birdsong-as-copyrighted-music/ <comment><USER2> This just happened to me with a parody video I created. I created all the music from scratch and obviously rewrote all the lyrics. And UMG review my video and said that it was indeed their property. So, I can not run ads against the video, which at this point has probably cost me $500-$1,000. There is nothing I can do right now but wait and hope YouTube changes its mind...

The video:

<http://www.youtube.com/watch?v=IJQAnamKeLs>

Here is what I see:

These content owners have reviewed your video and confirmed their claims to some or all of its content: Entity: UMPG Publishing Content Type: Musical Composition

These content owners have reviewed your video and agreed with your dispute: Entity: Music Publishing Rights Collecting Society Content Type: Musical Composition

Your dispute is still awaiting a response from these content owners: Entity: Social Media Holdings Content Type: Musical Composition

What should I do?

No action is required on your part. Your video is still available worldwide. In some cases ads may appear next to your video.

What can I do about my video's status? Please note that the video's status can change, if the policies chosen by the content owners change. You may want to check back periodically to see if you have new options available to you.

Please take a few minutes to visit our Help Center section on Policy and Copyright Guidelines, where you can learn more about copyright law and our Content Identification Service.

<comment><USER3> It's different in that neither party has claimed ownership of your content. If they delayed with intent to cause damage, sure, but otherwise a court would probably defer to the extra-judicial process Google is using. It's not ideal, and I'm not really sure if there are sites with similar revenue sharing deals like YouTube you could post it on, but it's how the cookie crumbles, unfortunately.

</comment></comment><comment><USER4> I was so furious after reading this article, I wrote to Rumblefish demanding an explanation.

Anyway, it's good news, I've just received this email.

Kristian,

Thank you for your note, just read your email and I share your concern. The YouTube content ID system mis-ID'd birds singing as one of our artists songs. We reviewed the video this evening and released the claim that YT assigned to us. One of our content id representatives made a mistake in the identification process and we've worked diligently to correct the error once we were made aware of it earlier today.

Thank you for voicing your concern. Very much appreciated. We're doing our best to improve the process as it's very challenging for our team to keep up with the massive amount of claims coming through which grow every day.

All the best,

Paul Anthony | Founder and CEO | Rumblefish

<comment><USER5> A ray of hope. Albeit a very small one in comparison to al. the other things that are very wrong with the system.

<comment><USER4> Absolutely, it's a shame that only cases with a little publicity get reviewed and corrected. Still, least there's a small positive to take away from this.

</comment></comment></comment><comment><USER6> What is truly appalling here is the claim that the video had been reviewed by humans, who had determined that birdsong was copyrighted music (although the birds ought to be flattered by that), and, as such, Rumblefish is either lying (about having looked at all), hoping to make a quick buck, or criminally incompetent. I wouldn't be surprised if Rumblefish was trying to make a buck or two off of ads here, but I'm guessing that that's their standard response, and they hope whoever made the video will give up.

<comment><USER7> Are there humans from Google involved in the review process somehow? Or do they just automatically accept claims from the copyright owners?

<comment><USER8> >Are there humans from Google

Yeah, good point, are there humans at Google? Or have their bots finally taken over?

<comment> loup-vaillant If they had, we'd already be doomed. <http://intelligenceexplosion.com/>

</comment></comment></comment></comment><comment><USER9> During the SOPA mess, a lot of people on this site were saying that the DMCA is fine, we don't need a new law.

However, it seems pretty clear that the DMCA is not fine. There need to be better protections against this sort of thing - no hiding behind "it wasn't a DMCA takedown notice" when your automated takedown bot fraudulently implicates someone.

Also, and this almost goes without saying, we need to modify the anti- circumvention provisions of the DMCA to at least legalize jailbreaking, whether the device is an iPhone, a PS3, or whatever. Though ideally the anti- circumvention provisions should be repealed wholesale, since they're unreasonably broad and create huge damages for a wide class of perfectly valid uses.

<comment><USER10> > There need to be better protections against this sort of thing

There is always a better protection against this sort of this: host your own content.

It is regrettable that as the Internet has grown, and services that offer hosting for free in exchange for attaching ads have grown, a lot of people have forgotten that you're under no obligation to host a video file on one of those services.

If your blog/pictures/videos/whatever are important to you, look for a self- hosted solution. If enough people are interested in doing that, businesses will appear to help the less technically literate do it as well, just as there are numerous businesses today that will make you a web site, register you a domain, and host everything on their servers but otherwise under your control.

Unless that starts happening, all the big video sites (not that there are many of them in the grand scheme of things) can impose more or less any terms they want, including claiming all kinds of rights to use any content you upload and not giving any guarantee that they will continue to host it or that they will not modify/corrupt/ad-splatter it.

</comment></comment><comment><USER11> I am pretty sure this amounts to fraud on the part of Rumblefish.

<comment><USER12> Why? Birdsongs are actually copywritable music, and since they all sound fairly similar it's certainly possible that the software confused his recording with something from the Cornell lab of ornithology, which licenses their bird recordings to Hollywood, toy makers, musicians, etc.

<comment><USER13> _since they all sound fairly similar_

That's not how copyright works. If you make an original work, it can be _identical_ to a copyrighted work and not be infringing. It's only a problem if you make your work from the copyrighted one.

<comment><USER14> I suspect Cornell do have mechanical copyright in some recordings of birdsongs - which'd allow them to charge royalties on anybody using copies of the recordings they made, but there'd be no composer or performer to own the actual "song". (Sort of like how you can't claim copyright in every photo of a sunrise just 'cause you took one, but you do have rights with regards to the particular sunrise photograph you shot.)

<comment><USER15> In GB the copyright definition apparently includes the mere idea of a photograph: [http://www.amateurphotographer.co.uk/news/photographers_face...](http://www.amateurphotographer.co.uk/news/photographers_face_copyright_threat_after_shock_ruling__news_311191.html)

<comment><USER16> In the U.S. as well.

"Infringement of Copyrighted Photographs Under U.S. Copyright law, you violate the copyright owner's exclusive rights of copying and/or to create a derivative work by creating a work that is a copy of or "substantially similar" to another's. The courts determine whether the two works are substantially similar by comparing them and evaluating whether copyrightable elements have been used in the second work. A court is much more likely to find an infringement if the subject of the photo has been "set up" by the photographer and contains creative and original elements, compared to a photograph of subjects that already exist, such as in nature or a structure such as the Golden Gate Bridge."

[http://www.photoattorney.com/2008/11/does-derivative-work- vi...](http://www.photoattorney.com/2008/11/does-derivative-work-violate- your.html)

The case you quote is not so easy, though.

The tough decision is if New English Teas intended to violate copyright, and if the Red Bus photos were original enough to claim copyright.

The judge decided that the Red Bus photos were original enough (creator claimed 80 hours of work), and New English Teas had used the original photos earlier on, and was made to remove them, so there was clearly intent to violate copyright. However, I'm not so sure if the Red Bus photos were original enough, or if there was intent to mislead buyers into thinking the New English Teas were Red Bus licensed.

</comment></comment></comment></comment></comment></comment><comment><USER17> So there are two things here.

The first is Google's algorithm incorrectly identified something as another work. This is bit is a bit "yeah, whatever". False positives will happen, what's important is the processes that are put in place around the algorithm to help resolve the errors.

And that's the second bit and the foobar bit, where a "human" check has confirmed it. Now, I may be skeptical but it feels very much like no-one ever looked at this, that Rumblefish basically automatically reply "yeah, that's ours" to any request or question and then leave it up to the poor video owner to show otherwise.

What I'd suggest needs to happen here is, at the very least, Google, Rumblefish or whoever need to state the piece of work that's being infringed. On Google's part this should be trivial - it's algorithm must have got a match against something specific which is cataloged.

The "copyright owner" at that point has a far more straight forward check (if they bother with it) and the video producer at least knows more about the claim rather than ending up in a slightly Kafka-esque position.

</comment><comment><USER18> This is a massive problem on YouTube right now. Essentially, there are a few large troll operations that abuse Google's "Content ID" system to steal ad revenues from random accounts, while Google shows no interest in fixing the system or shutting down the fraudulent accounts. The targetted videos often contain original music or no music at all. It's so common that it seems to be a fact of life for anyone as soon as they start monetizing their videos, and it's enough to remove any desire I have to do business with Google.

</comment><comment><USER19> Rumblefish CEO responded on Slashdot: [http://yro.slashdot.org/comments.pl?sid=2693545&cid=3916...](http://yro.slashdot.org/comments.pl?sid=2693545&cid=39168105)

<comment><USER16> And a Rumblefish employee (Lead Architect actually).

They at least appear to be answering questions, that's a start.

Edit: apparently an AmA on Reddit too. From <http://news.ycombinator.com/item?id=3640452>

[http://www.reddit.com/r/IAmA/comments/q7via/im_the_ceo_of_ru...](http://www.reddit.com/r/IAmA/comments/q7via/im_the_ceo_of_rumblefish_i_guess_were_the_newest/)

</comment></comment><comment><USER20> The question here is not about Youtube's algorithm being perfected or not, but about why they are having such an automatic censorship tool in the first place?

They aren't doing anything illegal by _not_ having the tool, which means they are doing it voluntarily, and since the tool is not perfect, Youtube itself can be more abusive than the copyright owners asking for takedowns. Google needs to stop this practice.

<comment><USER21> Because of all the lawsuits from media conglomerates.

<http://en.wikipedia.org/wiki/YouTube#Content_ID>

</comment></comment><comment><AUTHOR> The interesting part is that Rumblefish is the largest sync/track provider and has a vested interest when it comes to increasing copyright ultra- protectionism. Which is sad because you would think that there "process" as they call it would be the non-legislative answer to audio copyright billing.

5,345,377 Social Soundtracks Licensed(and counting)

</comment><comment><USER22> Geezus internet. How and why are so many people getting so damn angry about this?

Rumblefish presumably gets thousands upon thousands of copyright infringement claims every day. And so far, there has been one guy who had birdsong mistaken for music. Instead of just writing an email and getting it fixed, he decides to make a post about it to make people angry, and then once Rumblefish hears what happens it gets fixed.

Let's look at the key elements here.

1) It got fixed!

2) Of course they use an automated system for confirming copyrighted audio content. It's a second line of defense after Google's initial scan. The third line of defense is called customer support, and in any other situation that would be the obvious and trivial solution.

3) It's one guy! This happened ONCE!

4) It got fixed!!

And if you're still worried about your own stuff being falsely identified as copyrighted music, rest assured you can always make an internet s __*storm about it and your problem will be solved (this is clearly the ideal solution, yes?)

Now can we please get on with our lives?

<comment><USER23> _Of course they use an automated system for confirming copyrighted audio content. It's a second line of defense after Google's initial scan._

It's good to pay attention to cases like this, if only to flush other such cases out of the woodwork. You say it's once, but it's more accurate to say it's at least once.

I also say that if I were greedy, I'd make my "automated system" confirm everything as a violation. I'm not making that up as a crazy consume conspiracy theory. That's exactly what robosigning was: mass-production of claims that put the burden on consumers to figure out, and show evidence, of whether the company was making a valid claim or not. Robosigning (or simply returning "true" in this case) is extremely cheap, gets the job done in cases where the company is in the right, and gets the job done in many cases where the company is wrong because some people fail to jump through the hoops placed before them. In either case, it softens people up a bit before the company has to do any real work.

In the case of robosigning, it might actually have been profitable, even though they got caught at exactly the wrong time, when people were mad as hell about the crisis. At any other time, the penalty for getting caught would have been lighter, and it's certainly profitable when you don't get caught. If the strategy comes close to making money even in the worst possible case, I'd call it successful, and any established means of gaining an advantage at somebody else's expense merits a certain amount of vigilance. We put locks on our doors, we have laws against fraud, and in cases like this:

 _rest assured you can always make an internet s_storm about it and your problem will be solved (this is clearly the ideal solution, yes?)_

Exactly. I'm not any good at it myself, but I have to admit, it's the best way to get things done as a consumer.

</comment></comment><comment><USER24> Just more evidence that the DMCA is broken. This isn't YouTube's fault, it's a problem of Rumblefish harassing YouTube users.

<comment><USER25> No. On the face of it sounds like a problem of Rumblefish _stealing_ from users.

<comment><USER26> It's not accurate when the RIAA says stealing like that, and it's not accurate when anyone else does either.

<comment><USER27> The ads on the video pay out to Rumblefish instead of the video's owner.

<comment><USER9> The word "steal" still obscures the true nature of the problem, which is one of demanding services they have no right to based on a flawed law. That's not theft, though it could be described as fraud, extortion, or racketeering.

<comment><USER28> If you really want to be pedantic, "steal" is not a legal term. It's a term of common English routinely used to refer to _any_ of theft, fraud, extortion and racketeering.

<comment><USER26> I find it hard to equate an automated (bad) copyright takedown with any of those terms. Or file downloading, for that matter.

<comment><USER28> Exploiting an automated takedown system for the purpose of extracting advertising revenue from non-infringing content to which you don't own the copyright is undeniably fraud.

<comment><USER26> Fraud requires intentional bad faith. You can't accidentally commit fraud. (And there is no proof that this was anything but human error.. you could probably make a good case for gross negligence though.)

<comment><USER28> You can't know intent in either direction. That's an issue of determining _guilt_ , but this thread is a (ridiculous) digression on whether it's linguistically appropriate to use the word "steal" in reference to the takedown request. I assume you would therefore agree that it's valid to say that Rumblefish has _allegedly stolen_ the ad revenue. So therefore you agree, right?

</comment></comment></comment></comment></comment></comment></comment></comment></comment></comment><comment><USER29> has google published anything about how their audio copyright detection algorithm works? i ask because i know some people faced with the (clearly related, it seems) problem of automatically recognizing birdsong (as in - identifying from background noise, and figuring out what species).

<comment><USER30> Apparently they licensed it from AudibleMagic:

<http://www.audiblemagic.com/>

(source:)

<http://www.csh.rit.edu/~parallax/>

</comment></comment><comment><USER31> The CEO of Rumblefish did an AMA on reddit as a result of all the press this story received: [http://www.reddit.com/r/IAmA/comments/q7via/im_the_ceo_of_ru...](http://www.reddit.com/r/IAmA/comments/q7via/im_the_ceo_of_rumblefish_i_guess_were_the_newest/)

</comment><comment><USER32> The video in question: <http://www.youtube.com/watch?v=nPBlfeuZuWg>

Due to the low quality microphone the sound is a bit metallic. Perhaps that throws off YouTube's recognition software. But if there are a lot of false positives then Google should adjust its algorithm.

<comment><USER33> The author makes it sound like Rumblefish manually inspected the video after the author disputed the automated claim, and Rumblefish still insisted the audio was copyrighted.

</comment></comment><comment><USER34> It seems big companies fold to pressure from large companies and political pressure. It is sad they see a creator as less valid then a bigger company. I think they fear the legal teams of these larger companies. Many of them have legal teams waiting around to file suit and since they are salaried lawyers it costs the companies nothing extra to sue. On the other hand music companies already lost billions on free music being used. It appears every industry is either going through or will go through a lower cost or free model. Examples are books, music, newspapers, and the list goes on. Who pay 15 for a cd when it is easy to download for 0. I am happy many top music acts still make so much that it doesnt matter. Firemen, teachers, and cops make pennies compared to Snoop Dogg? Who deserves more? That's the dizzle for schizzle my nizzle!

</comment><comment><USER35> Small claims court. Sue them for damages (and punitive, if small claim courts allow).

Also, report Rumblefish for sending fake DMCA requests, it's illegal.

<comment><USER36> Rumblefish hasn't sent a DMCA takedown request, they've used Google's provided tool to say that they own the rights.

<comment><USER37> I don't know if the video was ever actually taken down, but it's available now. Either YouTube has ruled in favour of the poster or Rumblefish is taking a cut of the advertising on the video.

If it's the latter, then I believe that makes Rumblefish guilty of copyright infringement, since they're profiting off the copyright (the video) of the poster.

DMCA request or not, that seems like it would be actionable.

</comment></comment></comment><comment><USER38> Youtube is only complying with DMCA safe harbor by doing this. It requires a _speedy_ takedown of whatever content the claim was filed against. I'm getting slightly sick of people claiming this is Youtube's fault. YT is required by law to do this.

<comment><USER39> With that being said, it seems appalling that it is possible on Rumblefish side to _claim_ copyright on bird songs, assuming that they have vetted the audio personally.

Then again, should we not say that the algorithm that processes for copyrighted audio is imperfect and therefore could be improved further to not have false positive such as this?

I would bet that someone would have copyrighted sounds such as cash register's ka-ching!. Although use of such sounds from a specific recording should be copyrighted, I should be able to record a video of a real world cash register without anybody in the world claiming my work to be theirs.

<comment><USER14> "With that being said, it seems appalling that it is possible on Rumblefish side to claim copyright on bird songs, assuming that they have vetted the audio personally."

Two comments:

I wonder of Rumblefish intentionally err on the side of false positives, and use that stance in marketing their service to copyright owners? Cynical-me thinks perhaps this is just some viral marketing scam - with the birdsong being used on purpose since there's no legal-entity who could be said to be "the copyright holder" who could file fraud charges?

Secondly, I wonder if there's some recourse available to audit Rumblefish over this? If they're accepting payment from Google for their alleged copyright interest in the birdsong, how exactly are they disbursing that money and to which "copyright holder"?</comment></comment></comment>


Why I love Java - <AUTHOR> https://dev.to/acoh3n/why-i-love-java-5c14 <comment><USER2> This is a pretty empty and very generic article. The author likes Java because he's productive in it, there is automatic memory management and a large community.

Run s/Java/${SOME_LANG}/ over the text and you wouldn't know the difference.</comment>


Irwin Jacobs, Feared 1980s Corporate Raider, Dies in Apparent Murder-Suicide - <AUTHOR> https://www.wsj.com/articles/irwin-jacobs-dubbed-irv-the-liquidator-made-his-name-as-a-corporate-raider-in-the-1980s-11555424688 <comment><USER2> I used to love to bowl. (10 pins.) I wanted to be a professional bowler in my youth. I remember reading about Irwin Jacobs when he bought AMF, the giant bowling manufacturer.

The Bowler's Journal (the leading bowling industry publicaiton at the time) ran a cover story on Jacobs, complete with a tuxedo and high-rise office. Jacobs assured us that he was in bowling for the long term, bringing more resources to AMF so it could raise the sport to new heights.

Of course he broke up AMF shortly after, extracting profits from the pieces and pretty much ruining everything good about the company.

I didn't think well of him.

<comment><USER3> AMF is mentioned in the article, guess they had more businesses than bowling (like yacht construction). Still it was amusing reading the article thinking they were just swimming in bowling profits:

"In 1985, he acquired the conglomerate AMF Inc. and denounced the management for what he considered lavish spending on limousines, corporate jets and plush office space in White Plains, N.Y. "Their appetite at corporate was more befitting of General Motors," he told the Times."

bill-murray-kingpin.gif</comment></comment>


Key Advances in the OpenGL Ecosystem - <AUTHOR> https://www.khronos.org/news/press/khronos-group-announces-key-advances-in-opengl-ecosystem <comment><USER2> OpenGL is long overdue for a spring cleaning. A bit less than 10 years ago there was an initiative to create a new OpenGL API, code name "Longs Peak" which would have solved some of the obvious issues there are.

Right now there's a variety versions of OpenGL out there and they are incompatible in subtle ways. To write "portable" graphics code, you need both compile time and runtime version checks for a variety of features. Some restrictions in mobile versions make sense because of hardware requirements, others are just plain ridiculous.

And the versions of OpenGL that vendors ship is very diverse. For years, Nvidia and AMD have been the only ones to provide (at least almost) the latest version of OpenGL (but only on Windows and Linux, not Mac). Other vendors are lagging behind by several years.

I won't even start listing the obvious problems with the OpenGL API. Everyone who is working with it knows that the API is ridiculous.

I'd like to see an API clean up (ie. rewrite from scratch), a common shader compiler frontend, a common shader binary format and common tooling like benchmarking and profiling tools. Perhaps even a software emulated "gold standard" implementation.

At the moment, writing practical OpenGL applications is miserable. It's quite alright as long as you're working on a small project for your own enjoyment but once you have to start dealing with a variety of driver bugs from different vendors and whatnot, it takes a lot of time to actually ship an application.

<comment><USER3> Would you mind clarifying your problems with the API? I've done a fair amount of work with OpenGL (mostly ES), and I've found it to be a sensible low-level target for my rendering engine. That's not to say I haven't run into problems, but I assume any high performance graphics API has a learning curve. I don't have experience with Direct3D or other APIs to compare with it though.

All of my complaints with OpenGL are related to, as jarrett mentioned, the lack of debugging tools and the fiddly cross-platform and cross-version issues.

<comment><USER4> The craziest thing I found in OpenGL API was that there was no way to bind data to a primitive. You have to repeat it for every vertex of the primitive, which sometimes also means that you can't share vertices between primitives. This results in a 9x overhead in data for triangles!

Examples illustrating what I am saying:

[http://stackoverflow.com/q/6530700](http://stackoverflow.com/q/6530700)

[http://stackoverflow.com/q/6056679](http://stackoverflow.com/q/6056679)

[http://stackoverflow.com/q/23879737](http://stackoverflow.com/q/23879737)

<comment><USER2> You can definitely "bind data to primitives" and share vertices between triangles in OpenGL and that has always been possible ever since the first version of OpenGL.

Are you talking about legacy OpenGL style glBegin/glVertex/glEnd? That hasn't been the preferred way of drawing ever, and since OpenGL 1.5 (circa 1999), vertex buffers are the way to go.

If you want to share vertices between triangles, you should build and index buffer and use glDrawElements* functions.

<comment><USER4> I meant to say "bind data to elements". It's been a while since I programmed in OpenGL so the terminology is shaky.

I saw your other answer below too. I am not able to visualise how the duplicate data could affect caching postively. But anyway, I don't have deep knowledge about the hardware.

And I didn't know other 3D APIs had the same problem. Thanks for mentioning that.

</comment></comment></comment></comment></comment><comment><USER5> The quotes on that page feature the best job title I have ever seen: "Aras Pranckevičius, graphics plumber at Unity."

<comment><USER6> BTW Aras is a great guy to follow on twitter: [https://twitter.com/aras_p](https://twitter.com/aras_p)

<comment><USER7> Heh. [https://twitter.com/aras_p/status/496027538460651521](https://twitter.com/aras_p/status/496027538460651521)

</comment></comment></comment><comment><USER8> Direct State Access is finally in the core spec. That might be the last missing piece of what OpenGL 3.0 was supposed to be.

</comment><comment><USER9> We've come a long way from the time that the HP technical support person asked Steve Strassmann "Why would you ever need to point to something that you've drawn in 3D?"

[http://www.art.net/~hopkins/Don/unix- haters/x-windows/disast...](http://www.art.net/~hopkins/Don/unix- haters/x-windows/disaster.html)

<comment><USER10> In a twisted way that's still true though :) Getting the content of the pixel beneath the mouse still isn't trivial since CPU and GPU run in parallel and the CPU is at least one frame ahead. In general it's better to only push data into the GPU, and not trying to read any data back with the CPU.

</comment></comment><comment><USER11> Next Generation OpenGL: Key Advances That Will Bring OpenGL to Where Direct3D Was Two Years Ago.

<comment><USER12> Not quite. It will be like Mantle.

Mantle caused MS to rapidly announce DX12. So it's the exact same direction where everyone else is going.

While DX is more pleasurable to use than OpenGL (I personally am purely OpenGL dev, yet I acknowledge it's weaknesses) it has similar limitations. Very limited threading model and massive CPU overhead in drawcalls.

The future of graphics api's is going to be mantle style command buffer generation from multiple threads and then sending those buffers to the GPU.

</comment></comment><comment><USER13> HOLA FACU:D!</comment>


PyGTrends: Python API for Google Trends Data - <AUTHOR> http://www.juiceanalytics.com/openjuice/programmatic-google-trends-api/ Google Trends added functionality this week for CSV output of trends data. The code in the post is a quick hack that handles authentication and data munging, allowing you to query google trends directly from python.  Should help people get started building some neat trends mashups. <comment><AUTHOR> Google Trends added functionality this week for CSV output of trends data. The code in the post is a quick hack that handles authentication and data munging, allowing you to query google trends directly from python. Should help people get started building some neat trends mashups.

</comment><comment><USER2> Look super userful</comment>


Domains, codomains, ranges, images, preimages, inverse images « Gowers's Weblog - <AUTHOR> http://gowers.wordpress.com/2011/10/13/domains-codomains-ranges-images-preimages-inverse-images/ <comment><AUTHOR> Tim Gowers explains aspects about functions in a very clear way. Much of the stuff can be applied wholesale to many programming concepts. Every programmer, actually every job remotely involving math should really know this stuff.

As an abstraction tool the concept of function is powerful. It basically came into its modern form hand in hand with Calculus and not surprisingly was popularized by Euler.

Even as a programmer having a firm understanding of the function abstraction from Math will have real benefits in terms of reasoning about basic structure of code.</comment>


IsWhale – Find High-Value Users - <AUTHOR> https://iswhale.com/

<comment><USER2> I worry about things like this because I see them leading to the following situations:

\-----

1) Whale signs up for game with IAP

2) Whale's game difficulty is increased so they will spend more in IAP

\-----

1) Whale signs up for shopping site

2) Whale's prices have a slight markup because they know they can afford it

\-----

This is similar to how airlines will price flights differently based on your location and I think it was a travel site that was show OS X users higher prices for travel packages.

<comment><USER3> I'm pretty sure you just wrote their elevator pitch.

<comment><USER2> I'm afraid that you are right. I'd also love to hear from HN to how they feel about this... On one hand it's very interesting and what startup wouldn't want to know which of their users they should focus on on the other does this mean that non-whales don't matter and/or will we, unfairly, try and suck more out of whales.

I can see great benefits from being able to listen to whales (prioritise their bugs/issues/suggestions) it's a constant struggle when building a product to know what advice to follow but I worry that this means that people without money to blow will have their voices limited as a result which flies in the face of what I love about the internet which is: everyone is equal (now I know this doesn't work perfectly in practice but well enough IMHO).

At what point do we ignore users who are not whales or even kick them off our platform because they are a leach on our resources? This sets a dangerous precedent IMHO. I've wondered if one day they will be able to tag or detect people who never click on ads (and/or have "ad-blindness") and force them to pay to use resources supported by ads.

</comment></comment></comment><comment><USER4> In online-gambling a whale is a gambling addict. It's the kind of person who thinks the Martingale system is a good betting strategy. They are contrasted with another sea creature, the shark, which are persons who are good at finding games in which they have a small edge versus the house.

Those addicts are incredibly profitable. One addict can easily generate more in life-time profit than a hundred regular users.

</comment><comment><AUTHOR> Whales are sort of defined by the user of this service. Highly visible social accounts can make a whale but also social accounts that match terms. Finding those who sign up and are also interested in Linux or Mongo or ServiceX.

Identifying high-vis users, kinda like the Klout score, means I can make sure to focus on that one of 20+ daily sign-ups.

This could be used for pricing, I suppose but my intention when building was to identify tech-savvy users when they sign up. Also investors or those connected to them.

The back end tool is a little rough and as I expand it the Whale data will expand too.

</comment><comment><USER5> What makes a user "high-value"?

As a friend of mine just joked in irc, this is Klout-as-a-Service.

<comment><AUTHOR> Highly visible on multiple social networks. Likely to be an "ambassador" or "cheerleader". Can influence others, has a high infection rate for viral coefficient

<comment><USER6> That's not a whale. A whale is someone with a lot of money and willingness to dump it into your pocket when giving the right incentives.

I get what you're trying to do, but the terminology is not only wrong it's insulting to the users with whom it's most critical to maintain a good standing with. If anyone discovered I was doing something to influence their opinion of my apps based on the fact that a service tagged them as a "whale" i can pretty much guarantee they'd spread NEGATIVE influence about my products and myself.

<comment><AUTHOR> At casino's Whales know they are Whales and still love the special treatment. I think all humans like to feel special. As an entrepreneur I have difficutly identifying who is the most important to make feel special. All customers/prospects are not equal. EG: when an investor-class user signs up I want to know and make sure they get more than my programmed intercom.io message schedule. (also intercom.io FTW!)</comment></comment></comment></comment>


Universal basic income seems to improve employment and wellbeing - <AUTHOR> https://www.newscientist.com/article/2242937-universal-basic-income-seems-to-improve-employment-and-wellbeing/ <comment><USER2> People perform better when their brain isn't constantly buzzing from financial stress? Who'd've fucking thought...

<comment><USER3> It might be common sense to you and me but a lot of people doubt it. Studies like this are important.

</comment></comment><comment><USER4> [https://news.ycombinator.com/item?id=23103251](https://news.ycombinator.com/item?id=23103251)</comment>


Ask HN: How does HN's algorithm know if a post came from a YC company? - <AUTHOR> When YC companies post to HN, they go straight to page 1. Is this because of the account that posted, the URL of the post, or something else? <comment><USER2> This is false, the submit page is the same non-yc people use. They also do not go straight to page 1. There is literally no advantage other than

1\. putting "YC (s|w)(04..12)" causes users to notice quicker and upvote more 2\. the YC network itself.. batchmates upvote each other, and this voting ring boost is under constant combat from pg.

<comment><USER3> I believe he's talking about the job posts that don't allow comments.

</comment></comment><comment><USER4> I assume they have their own "submit page" that normal users don't see.</comment>


Apps Are Sharing Health and Finance Information with Facebook, Not Telling Users - <AUTHOR> https://www.theverge.com/2019/2/22/18236398/facebook-mobile-apps-data-sharing-ads-health-fitness-privacy-violation <comment><USER2> How much abuse will Apple and Google tolerate until FB gets deplatformed by the Play store and App store?</comment>


Company spamming me. Creative ideas to get them to stop? - <AUTHOR>

I've unsubscribed about 10 times from emails from Web Summit and even got human responses several times assuring me I will get no further emails from Collision Conference, Web Summit or any other events this company runs.
However, they continue to bombard me with emails.
It seems to be coming from an automated address that's probably not checked by any people (email@example.com) so what can I do to get the message across that I don't want to hear/have anything to do with these guys anymore? <comment><USER2> It depends on how much time you want to spend on it. I have had similar issues with other companies and I just setup filters to send them to SPAM.

I've unsub'd from xsplit.com emails four or five times now. It's very annoying and somewhat discrediting when companies disregard your unsubscribe request.

</comment><comment><USER3> Send a complaint to their email service provider, they'll either get rapidly brought into compliance or banned for abuse.</comment>


Startups Get Schooled At TechCrunch Conference - <AUTHOR> http://www.pehub.com/50497/startups-get-schooled-at-techcrunch-conference/

<comment><USER2> TechCrunch itself points out the primary fallacy of TC50 in the first place:

"One of the best things about TechCrunch50 is not what happens on stage, but what happens before companies even get there. Startups that don't even have a working demo or a product are forced to build one just to be considered." ([http://www.techcrunch.com/2009/09/17/techcrunch50-wrap-up- co...](http://www.techcrunch.com/2009/09/17/techcrunch50-wrap-up-congrats-to- all-the-startups-who-made-it/))

While I'm sure - and have seen, via the videos - many of the startups at the conference were thoroughly worked upon, the rush to present at TC50 doubtlessly hindered the quality of many of the presentations. Issues like "distribution and cost of sale," as Marc Andreessen noted, shouldn't be questions that a serious startup would stumble over...</comment>


Woman Endured 8 Days in Psych Ward Because Cops Didn't Believe BMW Was Hers - <AUTHOR> http://www.huffingtonpost.com/entry/kamilah-brock-nypd-bmw_55f2c9aae4b063ecbfa3e60d <comment><USER2> > Medical records also show that over the course of her eight-day stay, > personnel at the hospital repeatedly tried to get Brock to deny three things > before she could be released: that she owned the BMW, that she was a > professional banker, and that President Barack Obama followed her on > Twitter.

What the hell? Anyone who did this to her should, at a minimum, be fired and blacklisted from the medical profession. How long would it take to verify these things? Sure as hell not eight days.

<comment><USER3> This is a well-studied phenomenon of mental institutions: staff are acclimated to everyone in their presence being mentally ill and denying it. Consequently, they aren't exactly vigilant about finding exceptions. "The patient is lying and having delusions of grandeur" is probably an accurate assessment 99.999% of the time, so they are unlikely to doubt that assessment when the patient is in fact a researcher/journalist investigating the hospital.

<comment><USER4> This is the Rosenhan study ([https://en.wikipedia.org/wiki/Rosenhan_experiment](https://en.wikipedia.org/wiki/Rosenhan_experiment)), called one of the most influential psycholocial studies of all time.

</comment></comment></comment><comment><USER5> This is basically my nightmare. Nothing is as personally terrifying to me then being restrained and medicated.

I also worry about this kind of situation because my wife is black and we live in a very nice house and both enjoy nice vehicles. Neither of us (both tech people) enjoy dressing up-so were often poorly dressed. And we're both absent minded so we've both left our keys inside or had to fiddle with our cars in the driveway. I worry about how different an interaction with a passing police person could be for us-for me, an offer of assistance with a lost key, for her arrest and detainment.

</comment><comment><USER6> This is crazy. It would have taken all of 30 seconds to run the plates and verify the owner. This must surely have been intentional.

<comment><USER7> If she bought it via a loan from a bank, making the bank a financier. If it was paid off, it might show the bank still owning the car (ie not her) even though it was the bank's responsibility to transfer back, per the finance agreement. This has happened to me. The takeaway is that it's not as cut and dry as you would expect.

</comment></comment><comment><USER8> And I hope she wins her million dollar verdict and lives happily ever after.

</comment><comment><USER9> I'd suggest the fault is more with the police than the hospital... If police drop someone off and assert she claims to own a BMW but does not, claims to have a bank job and does not, then the hospital would have to assume she is delusional about those facts and make a diagnosis on that assumption.

<comment><USER10> Are those actual doctors or frauds?

If they can't tell a person that DOES have these things (bank job, BMW) from a dellusional person, then can they really treat dellusional people?

</comment></comment><comment><USER11> Its like a Terry Gilliam movie.</comment>


Ask HN: What are you using new programming languages for? - <AUTHOR> Hi HN,
It seems like a new language is being announced on HN every month or so.
I'm curious why there is a need for new programming languages, and what purposes they serve that existing languages aren't well suited for.  Do you have any experience you can share?
Companies I have worked for will always use technologies that have a large ecosystem and are battle hardened.  What is driving the need for new languages, and who are the early adopters? <comment><USER2> > I'm curious why there is a need for new programming language

In a way, this question is answered by your follow up question.

> and what purposes they serve that existing languages aren't well suited for

New languages are made because existing languages are not well suited for a given purpose. But, I don't think there needs to be a reason at all. The exercise of creating one is its own motivation and reward.

For me, I find a lot of the very popular languages to be mundane and boring to work with at best, and painful at worst. I would much prefer to work with Clojure instead of Java. Likewise, I would rather work with ClojureScript over JavaScript.

If I still want to work with a Lisp but I need something with a smaller footprint than Clojure I can reach for Janet or Guile Scheme. Currently I am using Janet for writing command line tools and I am exploring building webapps with Janet as well. I like the fact that it can produce statically linked binaries for the major operating systems and it is possible to port to others as well.

Although all of those are new languages they are all based on a very old concept (as far as computers are concerned), Lisp. Forth is another old concept that probably seems new simply because it isn't popular. However, I would rather use Forth than C or C++ to program my Arduino because it is interactive and I can program directly on the microprocessor.

Overall, I think a lot of companies and individuals choose what is popular because it is viewed as a safe choice. I prefer to explore the fringes because it is more interesting.

<comment><USER3> I don't agree. Mostly new languages are developed for the sake of creating a "new" language.

</comment></comment><comment><USER4> > I'm curious why there is a need for new programming languages

Exist a kind of self-defeating idea where "programming languages", "shell", "os", "editor" are in this "special" category of software that "must not be changed".

Is pretty weird. Specially from programmers: Why are we building new software? Anything we do, somebody have done before.

Somehow, is important to make the connection that our tools are alike _any other software_ , and we are like _any other user_.

If the regular user want/need/deserve new software, so as we.

\---

> I'm curious why there is a need for new programming languages

Because all of them sucks. For different reasons. Some have nulls, some have bad errors, some are crash-prone, some have dangling else, some are too obfuscate, some are too complex, or too simple, or slow, or take too much resources, or bad for imperative code, or bad for functional code, or bad for concurrency, or parallelism, or data transformation, or querying, or manage decencies, or scale....

Some are bad at many things at once ( _cough_ c _cough_ ). Few are good at many things. None, so far, is good at all.

Maybe is like the "theory of everything" and not exist "the perfect" lang. If that is the case, then we must chose "the best tool for the job".

Some langs, are picked NOT because are "the best tool for the _job_ ", but because are "the best tool for the _job recruiters_ " or something like that.

Despite that langs cause millions of dollars of losses.

You know how in certain niches exist that incredible, pathetic, terrible, software that also is the number 1?

And that thing is the bedrock of a billon dolar industry?

And you think, how is possible that humans allow them to use THAT?

And you think, how is possible that industries with $$$$$ "just not use something better?"

Yep?

That is our industry.

<comment><USER5> > Anything we do, somebody have done before.

I strongly disapprove of this thinking, it's simply a fallacy. There are constantly new problems created by technical advances, new tools and the evolving nature of everyday life and software.

Basic algorithmic solutions and libs like http in a specific language were "always there" and they're being redone forever. Things like controlling your light strip from your Switch are not, and there is always a need for such.

<comment><USER4> I fail to make it more obvious but this was the idea:

>Exist a kind of self-defeating idea .. (programing langs)

So

> Is pretty weird. Specially from programmers: Why are we building new > software (other than programming langs)? Anything we do, "somebody have done > before" (I must have put this in quotes!).

</comment></comment></comment><comment><USER6> Crystal lang. Web transported applications, as it was apparently intended. (HTTP requests being part of the std-lib and all...)

The syntax is like Ruby, but it's statically typed and compiled and the executable uses very little RAM.

It appears to be the sweet-spot (a GC'ed but typed/compiled lang) as I've yet to achieve high productivity on web apps with C or C++

FWIW it's worth, I gave Rust and Rocket a whirl. It seems nice. I didn't quite understand the borrow-checker... I'm familiar with RAII, ARC, pool-based approaches, trying to implement a malloc myself once (very badly) on an mcu... didn't get Rust yet...

One occasionally has to deal with Go, and it's getting rather widespread. Luckily, it's rather easy to transpose concepts to from your general algol family style stuff. I wasn't excited by Go, but it's not been frustrating either, and there's a LOT of stuff out there that has been getting it's "generational rewrite" in Go during the last few years... (wkhtmltopdf will be rewritten in trending languages every decade or so, right?)

</comment><comment><USER7> Crystal, to power simple cli applications that are easy to write (it is basically Ruby), fast (it is statically typed, and compiled), and easy to distribute (static binaries are supported, and Windows builds are possible.

<comment><USER8> Windows builds are possible? Can you tell me more?

I skipped Crystal because I absolutely need windows binaries.

<comment><USER7> You can't cross-compile yet, and it depends on WSL. I think proper windows support is one of the v1 (or soon after v1) targets.

</comment></comment></comment><comment><USER9> Go and Rust are two of my favorite examples. Go has a focus on minimalism and concurrency that makes backend engineering enjoyable. Rust is C without the memory problems and a decent package manager. Cool stuff for sure. I've recently built a Go-in-the-browser course for those who want to dip their toes in [https://qvault.io/go-mastery/](https://qvault.io/go-mastery/)

<comment><USER10> It looks good, I'm getting my feet wet but find the modules and importing my own code and correct project set up rather obscure, but the code writing is pretty straight forward. Looks like your course speaks to the latter but not the former: any advice about where to read up on the former?

<comment><USER11> I would say read this: [https://golang.org/doc/code.html](https://golang.org/doc/code.html)

</comment></comment></comment><comment><USER12> Nim

To practice coding interview algorithms and data structures problems. It is basically like Python but statically typed and compiled.

<comment><USER13> If you enjoy Nim, I encourage you to try Cython, which is more mature and enables much more practical applicability in most job settings. It's virtually never worthwhile for an employer to switch to supporting Nim if they already use Python and can just use Cython for targeted use of static typing in the small set of cases where it's useful.

<comment><USER12> I've tried it but I like the other features in Nim as well such as algebraic data types. As well, Cython isn't usually available in online coding environments while Nim is, at least sometimes.

<comment><USER14> Are algebraic data types supported? Can you do compiler checked exhaustive pattern matching on a sum type?

<comment><USER15> Yes, they are supported.

There are a few libraries for that, e.g. Patty, which IIRC supports exhaustive pattern matching.

Nim macros are as-powerful-as-Lisp-sans-reader-macros, and usually easier to write and use. A lot of things that require language revisions in most languages (e.g. async/await and pattern matching in Python/C#/JavaScript) are user-level constructs in Nim.

</comment></comment></comment></comment></comment><comment><USER16> Just because something is announced doesn't mean that everyone is using it, or even that it should be used. Many projects are created by people interested in the idea of writing languages, so they go off and do it then tell some people. They may be trying to solve a specific problem or just exploring some ideas, and other people might also join in to go along for the ride. It does not mean that they solved all programming language problems, or that it should be used as a general purpose language, and especially should not be pushed into production.

Don't mistake the number of announcements with people actually using these things. Many will last only a short time and then go away.

There are many other considerations you need to account for before you use a language for real work. The people constantly jumping from one fad to the next are not the ones you should be concerned with following.

</comment><comment><USER17> Learning Julia for doing scientific machine learning & numerical optimization. Faster to run than other interactive languages like Python or IDL, faster to write/iterate than compiled languages C/Fortran.

</comment><comment><USER18> At the time I started using Kotlin, I was a freelancer. My selling point was that I got things done 4x faster and at similar quality, but charged 2x the hourly rate as your average full-time programmer. It was a win-win for both me and my clients, but I had to keep doing better.

There were two bottlenecks to coding better. One was how much code I could "keep in my head". That translated pretty quickly to how much code I could keep within one laptop screen, or rather shortening method sizes to be no longer than my monitor screen.

The other bottleneck was Java boilerplate. We spent a _lot_ of time fixing this. With Android, there were bulky solutions like ActionBarSherlock. To put it one way, it was like writing articles in HTML, and we'd find Markdown-style hacks.

Default code would be for example, CodeView X = (CodeView) findElementById(R.id.codeview), when it could just be CodeView X = codeview. Multiply this by every single element on the page. Imagine doing enterprise forms for a city council (which was my job at the time). I was happy to spend non-billed time looking for solutions to this so that I can do better during by billed hours.

It was made worse that Android advancements drastically increased the length of code, with new things like RecyclerView/Adapters and Fragments.

Kotlin solved all these problems and very elegantly. It was simply part of the code, no special hacks needed. It did everything you'd need from Java. And there were things like functional programming and null safety which was a nice bonus.

Similar reasoning applies to Swift.

</comment><comment><USER19> Crystal has completely taken over my personal work. I have three projects in the pipeline (none public yet) that are mostly written in it: a 2D pixel art adventure game (desktop), a heavily scriptable email client (web service), and a sheet music reader/manager (largely a hardware project as well, targets a raspi hooked up to an e-paper display). The language has been such a breeze to use for all three.

</comment><comment><USER20> Cuelang is an example. It's for configuration where we really need a dedicated and common language, because Yaml is being abused at this point.

Golang is an interesting example as a language which decided to be minimal, opinionated, leave out fancy language features. It was built to be "in the service of software engineering."

Largely, the world evolves, tech evolves, and so will out languages for translation between the two.

</comment><comment><USER21> Rust. Embedded. Firmware from Scratch. Satellites/cubesat/space related.

<comment><USER22> How did you approach this in your workspace and successfully steer people away from C and C++?

I have successfully avoided C and C++ my entire career, for good reasons, but I am getting more and more interested in doing more true embedded system type of work. Rust is certainly the leading candidate for me.

</comment></comment><comment><USER23> Hi, right now I'm learning Go (to write an access control system for our makerspace). For me this is a pretty new language but currently at the point where I see quite large community and available libraries (wait, it is called a module, isn't it?).

Why? I love C language (static typing, you can shoot yourself in a leg == you can do almost anything, etc..), but C is not suited to write this kind of application. I also like PHP - large community, libraries, large OS systems written in it... But I miss some C features and getting bored by having dozens of dependencies for a simple project. Python - is slow (yes, can be tuned somehow, but brings more work to do) and I do not like the syntax and I miss brackets. For me, Go seems like a way to go from now on.

So, to sum it up - for me a new language is a way to solve some set of problems much easier (and sometimes more efficient and/or secure) than with an existing language. I wouldn't be happy if I had to write anything larger than hello world in assembly :)

<comment><USER24> > you can shoot yourself in a leg == you can do almost anything

That doesn't really follow. A language with plenty of undefined behaviour => shooting yourself in a leg. A language with plenty of defined behaviour => you can do almost anything. C falls somewhere in between.

</comment></comment><comment><USER22> To learn, have some fun, and think differently and more optimally. Most of the older and more mainstream programming languages like Python, Java, C, C++, etc. are simply uninspired. They have very mainstream and convoluted ways of thinking that bind you in ways that I personally can't take.

When I program, I try and think of the problem and possible solutions in the same way that I would if working with a human colleague. That is, I design systems in a human way. I try to program in such a way that maps to how I would explain the domain to someone else and not how the computer thinks of it.

Thus, I primarily think in terms of data, transformations on that data, interactivity, and concurrent or parallel operations. This approach almost chooses languages for me. My primary languages I am learning and using at home right now are F#, Racket, and Elixir. These are new (-ish) languages with new ideas based upon the strong foundation of old languages that had revolutionary ideas, so it's not surprising I landed on these, for me personally. Also, although each one is a complex language, they each have a simple core, which is another plus. Then there are visual languages like TouchDesigner and LabVIEW.

However, the curse is finding a place that uses the road less traveled because it requires the company and its employees to have made the non-standard choice and to consider the tools as important as whatever else they're working on. There are few places that use newer languages, but I am trying to become more agressive in just using them at my work. If they solve problems in an efficient way, you shouldn't necessarily need to ask for permission. People can be trained. Nobody asks for permission when they use Python or C or whatever else. They use it because that's all they know. The other way is to merge your personal interests with work by creating your own company, but that is an obvious high hurdle.

When using new languages, there is the problem of ecosystem, particularly in libraries. However, I am of the opinion that this isn't as big of a problem as often reported. It's a problem but not the no-go problem it is usually presented as.

</comment><comment><USER25> Programming languages, being engineered constructs, take a multitude of shapes.

It's not different in that respect from any other popular mode of software exploration like graphics programming or small webapps.

The more relevant question is with what makes a language "industry ready", and the answer there is mostly libraries, tooling and positioning. C is crude for many tasks, but its conjoinment with Unix gave it a leg up. JavaScript is the same way with the Web. More recently we have had some waves of corporate- backed languages in Java, C#, Swift, and Go.

New languages fall out of just about anything that has a data structure with control flow constructs, though.

</comment><comment><USER26> Dart

For building rapid GUI applications in Flutter

<comment><USER27> Noticed the other day Canonical and Google teaming up:

[https://ubuntu.com/blog/canonical-enables-linux-desktop- app-...](https://ubuntu.com/blog/canonical-enables-linux-desktop-app-support- with-flutter)

I have many cli tools that could use front ends ;)

</comment></comment><comment><USER28> Lean [1] [2] [3]

To try and revitalize my math life a bit... There are some things I quite miss, and learning Lean was my idea to force myself to get back in the swing of things.

[1]: [https://leanprover-community.github.io/](https://leanprover- community.github.io/)

[2]: [https://wwwf.imperial.ac.uk/~buzzard/xena/natural_number_gam...](https://wwwf.imperial.ac.uk/~buzzard/xena/natural_number_game/)

[3]: [https://www.youtube.com/watch?v=Dp- mQ3HxgDE](https://www.youtube.com/watch?v=Dp-mQ3HxgDE)

</comment><comment><USER29> We are using Go for all of our backend services and business logic. We needed something that is statically compiled, easiliy deployable and sufficiently fast.

</comment><comment><USER30> We're using Go for writing a kubernetes operator to deploy our product.

Rest of our stack is in Scala - primarily since it is very good for writing compilers - we have a few in our product.

We're a data engineering product - think Spark IDE.</comment>


Introduction to Information Retrieval  - <AUTHOR> http://nlp.stanford.edu/IR-book/information-retrieval-book.html <comment><USER2> I'm reading this now. Things like this remind me of why I love the Internet. So much great knowledge, freely available for everyone. It boggles the imagination sometimes.

Also, another book that some HN'ers might find interesting is this one:

Information Theory, Inference, and Learning Algorithms:

<http://www.inference.phy.cam.ac.uk/itprnn/book.html>

<comment><USER3> Another one:

The Elements of Statistical Learning: Data Mining, Inference, and Prediction.

<http://www-stat.stanford.edu/~tibs/ElemStatLearn/>

</comment></comment><comment><USER4> Related course: <http://cs276.stanford.edu> and syllabus: <http://www.stanford.edu/class/cs276/cs276-2009-syllabus.html>

</comment><comment><USER5> This is a fantastic book, I'm using it to build my own IR system and it has been immensely useful. I also own Managing Gigabytes but I find this book to be much more useful and up to date.

</comment><comment><USER6> Another +1 on this fantastic book. It's extremely accessible and concise while covering quite a lot of ground on the subject.

</comment><comment><USER7> This is a fantastic survey book. It's one of those few books that's exactly what it's called. You can always dive deeper on the stuff that's interesting/relevant to you after reading this book, but I have yet to find a book at this level that's this good.</comment>


Dat Desktop, a peer to peer data sharing app - <AUTHOR> https://datproject.org/blog/2017-01-10-dat-desktop-is-here <comment><USER2> And here are the actually interesting parts:

[https://docs.datproject.org/how-dat-works](https://docs.datproject.org/how- dat-works)

[https://github.com/datproject/docs/blob/master/docs/hyperdri...](https://github.com/datproject/docs/blob/master/docs/hyperdrive_spec.md)

Seems very similar to BitTorrent, but without trackers, with Content Defined Chunking (instead of Fixed Size Chunking) and utilizing Merkle trees + public key cryptography to enable append-only operations on shared data sets (rather than 'read-only once published').

<comment><USER3> Thanks for the links, here are a few more you may find interesting:

In progress paper - [https://github.com/datproject/docs/blob/master/papers/dat- pa...](https://github.com/datproject/docs/blob/master/papers/dat-paper.pdf)

Dat protocol site - [https://www.datprotocol.com/](https://www.datprotocol.com/)

</comment></comment><comment><USER4> If this lets me download specific files inside a shared folder instead of syncing the entire thing it would probably be the best sharing tool so far.

<comment><USER3> Yes! Dat protocol supports syncing only the data you want, including storing all history or only the latest content. We just need to get the UI added for this in the desktop app & CLI.

</comment></comment><comment><USER5> What are the differences between this and librevault?

<comment><USER3> Not very familiar with librevault, but on a cursory glance the design seems fairly consistent (p2p, encryption, offline support).

However, Dat has put a lot of focus into developing an ecosystem to build applications on top of the protocol. This is making awesome things like Beaker Browser[0] and peer-npm possible [1].

[0] - [https://beakerbrowser.com/](https://beakerbrowser.com/) [1] - [https://github.com/noffle/peer-npm](https://github.com/noffle/peer-npm)</comment></comment>


How Congress might crack down on H-1B abuse - <AUTHOR> http://money.cnn.com/2017/01/06/technology/h1b-reform-bill/index.html?sr=fbCNN010617h1b-reform-bill0804PMVODtopLink&linkId=33098713 <comment><AUTHOR> I like this on one hand, but on the other hand, it blocks new international college graduates from getting any kind of foothold in the American workplace. I wasn't underpaid out of college as a h1b worker, but I would never have made it to my green card if this law had been effect, and I wasn't part of an outsourcing firm or underpaid at any job I ever held.

</comment><comment><USER2> Laws created by people who have 0 skin in the game. Completely arbitrary and nonsensical.</comment>