John Carmack on expert witnesses and 'non literal' copying - <AUTHOR> https://www.facebook.com/permalink.php?story_fbid=1913546895546485&id=100006735798590 <comment><USER2> I don't have the evidence so I can't make a judgement about whether Zenimax or Carmack is in the wrong here, but this does point out something strange in our court system and that is where a "jury of peers" which translates to normal people with little to no knowledge of the underlying subject matter.

As our society becomes more specialized it seems a bit absurd to have people make judgements mostly based on how well each side can make incredibly complex things comprehensible and convincing without any real understanding of the underlying principles in the field.

It seems likely that the best expert witness in this case is not the most correct or credentialed, but the most charismatic.

It would be interesting to see how some of these big software trials (Google v Oracle) would have come out if the jury were made up of people who were both impartial and familiar with the industry.

<comment><USER3> This is a very common argument against juries, and I think it's one of the most powerful. Of course if you start to fill a jury with people competent in the area being judged, you run different risks, such as bias within the field. And it's still not clear that it would help with delivering justice.

But if we take the idea of a jury to be about judging by one's peers, it should be patently clear (heh) that the jury ought to be composed of the _peers_ of the defendant. As to how that works when there's a corporation on trial, I don't know. I have the funny image of Google and Oracle being judged by Microsoft, IBM and the FSF in the jury box :)

<comment><USER4> I once espoused a similar opinion, but have since changed my mind.

I'm a physicist, and once expressed to a judge I had met on the trail that I wished I could be judged by my academic peers. It was my belief that they would be more receptive to logical argument, understanding in their analysis, and more committed to fundamental principles. The judge was visibly surprised, but said little.

What later changed my mind was the realization that our society is a human one, and that humanity has a huge spectrum of experience. A group of physicists may not understand something about the world that others do. Furthermore, any select group of citizens will differ in belief somewhat from the interests of society at large.

But do I want educated and principled jurors on any jury that judges me, ones with enough fortitude to mis-trial a jury or nullify a law if that's what is required? Damn straight.

If the notion that your citizen-peers might not be informed troubles you, take time to help others form informed opinions.

<comment><USER5> >The judge was visibly surprised

I would be too. What you said is rather arrogant. If you had based it on their having specific physics knowledge, that would be one thing. But to say that logic and reasoning is the exclusive realm of academia---that's very insulting to a lot of people.

<comment><USER6> Insulting. But doesn't make it any less true or false. To presume that your average joe has bothered to play in the realm of academic, diligent logic and reason is a mistake. I am not saying you need to go into a university, I'm not claiming a piece of paper will lend you credibility in this regard. It's about the time and effort that people spend practicing these subjects.

How many people do you know outside of STEM that can comment on the characteristic of falsifiability of scientific theories? How many even bothered to take philosophy 101? How many have bothered to seriously invest themselves in substantive topics that will give them the tools and experience they need to make sound judgements in an array of different subjects?

We are talking about a society where in a large segment of the population does not believe credible scientific theories, and tend to latch onto what can and should be generally discarded viewpoints.

We (as a society) by and large do not hire people without basic math and reasoning skills; why do we suddenly pretend that these same qualifications should not be held in the justice system, or even entertain the idea that these qualifications should be raised to a higher standard. After all, juries sometimes have the power to ruin entire lives and families.

People's egos are not great bedrocks for their credibility and judgement to sit on. Let them be insulted.

<comment><USER7> Do we have evidence that STEMers are paragons of logical reasonable minded virtue outside of their particular domains?

<comment><USER8> You're getting downvoted, but over the last decade I've seen a pretty high number of STEMers that seem to confine their logical reasoning skills to very specific domains.

Example (I'll let you be the judge of which one is being irrational): I know STEMers who are adamant atheists. I also know STEMers who are pretty devout and think that science supports all of their beliefs.

For a less controversial example just look at the sheer amount of fashion trends that sweep through the programming community. Pushing those trends is literally a core function of this website. Just like many people in america jump from fad diet to fad diet , and we jump from fad framework to fad framework. The number of software engineering decisions we make that are based on actual software engineering research is vanishingly small.

Fellow STEMers please understand that you (and me) are human and very likely just as irrational and emotional as any other group of people. Ignoring that fact will lead to a false sense of superiority.

Theoretically science is supposed to be that paragon of reasoning where we attempt to falsify our beliefs and thereby make the more correct ones stronger, but my interaction with the academic world has made me realize that science is again mostly about developing relationships so that your papers will be favorably reviewed, publishing junk to make sure your boss thinks you are productive, and researching whatever the people paying your bills are interested in researching.

<comment><USER9> I agree with almost all of what you're saying, but the question isn't whether or not STEMers are very rational people. The question is whether they are better equipped than the average person to understand and scrutinize a logical argument that is presented to them in court.

Being better at that than the average population isn't a very high bar. Yes we make a lot of irrational choices and often times we will defend them in pseudo rational terms. Flame wars abound.

But the meta debate about the standard of proof for our claims is never far away. It counts for something to be able to come up with some sort of test or empirical support. Every front page story about medical experiments inevitably has a comment thread about correlation vs causality and various sorts of possible biases.

Do you really want to be judged by people who never had these sorts of debates and never had to defend their own arguments against this sort of criticism?

</comment></comment></comment></comment></comment></comment></comment></comment><comment><USER10> Interesting rant on expert witnesses. Mostly interesting because I've done some expert witness testimony and found it fascinating.

I completely resonate with John's issue with the expert's testimony and his understanding of it. John mentioned that their own expert testified differently but he didn't say whether or not their lawyers attempted to impeach their expert with cross examination. In the two cases I participated in, both sides agreed ahead of time on how to "handle" experts (in terms of cross examination). In one it was documents only (no testimony) and in one I was deposed by the opposing counsel after documents in which the lawyer clearly had some notes (presumably from their expert(s)) and was trying to get me to recant or change some of my points (I didn't need to).

In a jury trial I can imagine that having the jury understand your testimony is probably the most challenging.

And, like John, I consider the notion of 'non-literal copying' to be pretty ridiculous. In the limit it means "you read this code, understood how it worked, and wrote new code that could do the same function." You can stretch that to cover anything you have ever seen. Which is sad.

<comment><USER11> > In the limit it means "you read this code, understood how it worked, and > wrote new code that could do the same function." You can stretch that to > cover anything you have ever seen. Which is sad.

I thought the industry adopted "clean room" techniques decades ago to avoid this kind of appearance of impropriety.

It's more expensive, sure, but isn't eliminating legal risk something that typically gets paid for?

<comment><USER12> This particular point worries me. Are we now not going to be able to make a clean room re-implementation of concept? Do we need to worry about our previous employer suing us because we are successful?

<comment><USER11> They didn't seem to do clean room here. Seems like Carmack was wrote code for both companies which is the exact opposite of clean room.

(Please correct me if I misunderstood the facts of the case)

<comment><USER13> The employer of John Carmack is buying his work product for as long as he remains an employee. The company cannot buy his knowledge and expertise from him, as that is an integral and indivisible part of him.

So if you hire Carmack to write a rendering engine, then he quits and goes to work for another company, to write a different rendering engine, he is not "copying" the engine you bought. His memories of all the software he had previously written do not belong to you. You cannot contractually obligate him to forget a part of himself that makes him valuable as a software professional.

The work of a software professional is constrained, because while a given mathematical problem may be solved in an infinite number of ways, only a few of those ways will be optimal for any given set of computing conditions. Different implementations may be grouped by algorithm. With respect to computer graphics, all rendering engines will tend to converge on the fastest algorithms for painting pixels on rectangles. Because of this, _all_ rendering engines will be very similar in some respects. But they will also markedly diverge in others.

There are pretty much only 4 types of algorithm: the fast way, the frugal way, the magical way (viz. both fast and frugal, but requires top-level expertise to invent or understand), and the hardware exploit. Carmack is certainly capable of writing magical algorithms. And he can remember how to make the pixie dust. He will also tend to write all his implementations in a similar style.

Thus it is very possible that one person could write identical code at different times, while having no specific memory of the previous time it was written, nor any need to refer to associated documentation around that instance.

The clean room is clean if no _property_ of the other company was ever inside it. Carmack cannot be property, therefore his presence is not a contaminant. My opinion is that any non-disclosure agreement Carmack may have been a party to can only meaningfully cover the contribution of _other_ individuals to the project, since _Carmack-qua-Company-A-employee_ cannot meaningfully wall off knowledge from _Carmack-qua-Company-B-employee_. If Company A wanted to keep Carmack's raw knowledge and expertise from Company B, they would have had to get him to sign a non-compete agreement (which no one at his level of clout should ever do for less than a significant ownership share in Company A).

<comment><USER14> > The employer of John Carmack is buying his work product for as long as he > remains an employee. The company cannot buy his knowledge and expertise from > him, as that is an integral and indivisible part of him.

This is the entire point of non-compete agreements. To deny competitors the expertise the person has developed during their time at your company.

<comment><USER13> I'm not entirely certain that you read all the way to the end of my reply.

To really lock someone up, you need both a non-disclosure and a non-compete. And anyone worth a non-compete should make it really expensive for the company that is asking for it. You're trading away potentially several years worth of lost opportunity costs to basically _not be what you are_ for some time after leaving a company.

</comment></comment></comment></comment></comment></comment></comment><comment><USER15> It's kinda hard to square John's post-trial comments "I never tried to hide or wipe any evidence, and all of my data is accounted for, contrary to some stories being spread." with ZeniMax's post-trial comments at [http://www.gameinformer.com/b/news/archive/2017/02/01/zenima...](http://www.gameinformer.com/b/news/archive/2017/02/01/zenimax- awarded-500-million-in-oculus-lawsuit.aspx) that " (vi) Carmack intentionally destroyed data on his computer after he got notice of this litigation and right after he researched on Google how to wipe a hard drive—and data on other Oculus computers and USB storage devices were similarly deleted (as determined by a court-appointed, independent expert in computer forensics); ... (viii) Carmack filed an affidavit which the court's expert said was false in denying the destruction of evidence; "

What is a reasonable explanation of this discrepancy? Varying notions of what consitutes "evidence"? Or differences in facts?

<comment><USER16> > Carmack intentionally destroyed data on his computer after he got notice of > this litigation and right after he researched on Google how to wipe a hard > drive

Somehow I think John knows how to wipe a drive...

<comment><USER17> Every dev knows how to wipe a drive. Buy an identical drive. Throw old drive into the incinerator.

<comment><USER18> Lack of history is a clue system was wiped or reinstalled from scratch.

Its very difficult to make windows appear to be an old actively used install. Encase (or new software, I havent done forensic for >6 years now) will tell you everything about timestamps and what that means.

<comment><USER17> I was joking. But honestly, if you don't your secrets to be found, you do have to go for extreme measures.

</comment></comment></comment></comment></comment><comment><USER19> I suppose many weird things happen in $500m cases, but it'd be really unusual to have _trial testimony_ sealed against the agreement of the parties. Pretrial reports and depositions, sure. Media was apparently there reporting during the expert's trial testimony, and I'd be surprised if they cannot buy a transcript from the court reporter. ([http://uploadvr.com/court-oculus- zenimax-last-day/](http://uploadvr.com/court-oculus-zenimax-last-day/))

A quick look at the docket shows the expert witness at issue is likely to be David Dobkin of Princeton. That name appears on the docket and in media reports. Let me know if that violates HN norms, but expert witness trial testimony _is_ part of the public, permanent record in almost every case. If a witness is excluded for using unreliable methods, his or her value as an expert is ... diminished.

<comment><USER15> Interesting. David Dobkin is the Dean of Faculty at Princeton University who used to be a Computer Science professor there for a very long time before being Dean; his specialty was 3D geometry (not a 3D shader guy, kinda a bit more math-y than applied.)

I don't know if he ever presided over code plagarism cases with clever Princeton CS undergrads but since he was department chair for a fair chunk of time, information on that may well provide interesting insight into his methodology and previous experience. (Reviewing papers for various computer graphics entities might also contain additional plagarism experience but that's unlikely to be code-comparison-type plagarism.) Given his CV I would be very surprised whether he has considered abstract syntax trees much since he got his PhD 43 years ago, but I am not convinced by Carmack that is automatically exculpatory.

I would consider him a reasonable pick as an expert in 3D geometry and algorithms (although he hasn't been active in the field in 10+ years he did it for a /long/ time which is likely sufficient) and he may have dealt with plagarism of regular text during his career a moderate amount, but he does NOT seem to be an expert in code-plagarism or code similarity. It's interesting that the former is good enough and they didn't need an expert in the latter.

In any case, given the positions he's been at, whatever he's getting paid, even $600+/hour, I doubt he /needs/ the money. (If he is in fact the expert witness.)

<comment><USER15> Seems likely it was David Dobkin: [https://www.reddit.com/r/Vive/comments/5q5rf2/software_exper...](https://www.reddit.com/r/Vive/comments/5q5rf2/software_expert_claims_oculus_copied_code_from/) [https://www.elys.com/people/david-dobkin](https://www.elys.com/people/david- dobkin)

Following up on other posters' comments (thank you!), while Carmack proposed Abstract Syntax Trees based copyright detection, he didn't seem to appreciate the standard court methods for detecting non-syntactical copyright violation for code, the abstract filtration comparison test ([https://en.wikipedia.org/wiki/Abstraction-Filtration- Compari...](https://en.wikipedia.org/wiki/Abstraction-Filtration- Comparison_test)). Dobkin it seems didn't have to rule based on his own opinion or based on some academic method whether there was copying, but whether -- given the very specific method laid out by legal precedent to know whether there was copying, assuming that method was true -- whether when he applied it, it indicated there was copying. That makes a lot more sense to me. Dobkin would be a perfectly suitable expert for doing that sort of test.

I am a bit surprised Carmack didn't seem to recognize this basic point. But perhaps I am missing further subtleties. Or perhaps he is just trying to indirectly shift blame to Dobkin when it really belongs to the legal system (or where actually the legal system correctly discerned something was going on).

For the actual 90-page Zenimax ruling which describes the abstract filtration comparison test in the words seemingly given to the jury for this particular case, see pages 21 onward (but especially page 26-30) [https://www.scribd.com/document/338147994/Ruling-in- Zenimax-...](https://www.scribd.com/document/338147994/Ruling-in-Zenimax-vs- Facebook-s-Oculus)

For direct copyright infringement, only Oculus was found guilty (ZeniMax claims post-trial at [http://uploadvr.com/zenimax-responds-john-carmacks- facebook-...](http://uploadvr.com/zenimax-responds-john-carmacks-facebook- note-experts/) some employee admitted he copy-pasted some code to an Oculus SDK.) Carmack was innocent.

For vicarious copyright infringement (ie management should have known/prevented it, [https://en.wikipedia.org/wiki/Vicarious_liability](https://en.wikipedia.org/wiki/Vicarious_liability)), Oculus/Facebook/Carmack were innocent but Palmer Lucky and Brenden Iribe (before they founded Oculus?) were guilty.

For contributory copyright infringement ([https://en.wikipedia.org/wiki/Contributory_liability](https://en.wikipedia.org/wiki/Contributory_liability), a "party materially contributes to, facilitates, induces, or is otherwise responsible for directly infringing acts carried out by another party." ), Oculus/Facebook were innocent but Iribe, Palmer, and this time Carmack were guilty.

BUT, the ruling then goes on to say that "the defenses of license or de minimis use bars Zenimax's claim for copyright infringement" for Palmer, Iribe and Carmack (but not for Oculus). So whatever Palmer/Iribe/Carmack did wasn't that bad, but the actual copying of code to the SDK by Oculus somehow merited $50m damages (I can't see why that won't get reduced substantially on appeal, no copy-pasted-code-to-an-SDK really caused Zenimax $50m damages.)

For violating the NDA Palmer Lucky signed before Oculus was even formed (but which was ruled to still be in effect after Oculus), Zenimax gets $200 million.

And for "false designation of origin", ([https://en.wikipedia.org/wiki/False_designation_of_origin](https://en.wikipedia.org/wiki/False_designation_of_origin)), where the manufacturer/seller lies about the origin of their products, Oculus was guilty for $50m, Palmer Lucky (the visionary founder) for $50m and Brendan Iribe (the CEO) for $150m. I'm not sure I understand the factual basis of what happened here, but having been in startups that deliberately obscured the origins/key parts of underlying technologies to angel investors, I can see the origins of product misrepresentations as being worthy of substantial damages as it really does affect Oculus's value and Zenimax's ability to be compensated for "their part". Probably will get knocked down too, but this is the most defensible part of the $ amount.

Sorry for the long post.

<comment><USER20> > Sorry for the long post.

Don't be. This is the first post I've seen that actually answers more questions than it provokes. Thanks for helping me understand what happened.

</comment></comment></comment></comment><comment><USER21> The problem for John Camack was I think he was on both sides of the table writing code - so he wrote the code at Zenimax & re-wrote the code at Oculus. From a layman's perspective, he very well could have been "copying".

Normally when you design clean room implementations, you use different people and make sure that the people who write the code never see the prior written code and the person giving the spec doesn't see the newly written code.

They were just in-sufficiently cautious legally, possibly un-surprising given Carmack's prior history in founding Id software, and got hit with a lawsuit.

<comment><USER22> That seems like it is saying that Zenimax owns not only the code Carmack wrote while he was there, but also the knowledge and experience he gained while working there.

<comment><USER23> I think it's more that Zenimax thinks it owns the unique ideas that Carmack, a top talent in his field, created while he was working there. They're not angry that he wrote the same 4x4 matrix inverse function for both them and Oculus.

Or more specifically, if Zenimax's claims are correct, while he was working on company time and equipment.

<comment><USER14> What is the alternative? Do we force everybody to undergo mind wipes whenever they leave the company so they don't take any skills they picked up on the job with them?

How many people can forget everything they know every time they get a new job?

<comment><USER23> I think it's a balancing act with plenty of gray area. Personally, I almost always side with employees in cases like this, because it's ridiculous to me that a company could claim ownership of things that its employees create which are unrelated to their jobs, or which they produced with their own time and resources.

But I also think that there are some people who are exceptional cases. The real high earners in a company, especially if they're technical staff, are probably real luminaries in their field, and the company is paying them a significant premium in both money and autonomy because they believe that such an employee will produce a lot of valuable new ideas for the company.

So if your rock star programming wizard quits to start a new company, taking a project that they've been iterating on during work hours on their company computer for the past 6 months while they were drawing a paycheck...well, if you look at it from the company's point of view, you can understand why they would feel like they got robbed blind.

</comment></comment></comment></comment></comment><comment><USER24> This struck out at me:

> There are objective measures of code similarity that can be quoted, like the > edit distance between abstract syntax trees ...

If this became the primary legal metric, then programmers who stole code would change the code so that functions achieved the same output with practically no AST similarity. That is, they could maximize _functional_ similarity while minimzing _code_ similarity. This would be a large task, but one that might be easier and preferred to writing code from scratch, since you drastically reduce the trial and error process of writing code in a field where best practices aren't currently known.

But, while it's possible to game any 'objective' metric like AST distance, maybe that's preferable to the subjective and incentivized claims of an expert witness, who will be biased in favor of the party paying them. And while it's easy to get lost in the paragraphs of expert witness testimony, a collection of objective metrics is easy to compare to previous cases.

<comment><USER25> If you're genuinely mutating the AST, you'll end up with a different program that does something different or performs differently. There's only so many ways you can write a loop and they all look loop-like, whether it's for / while / do / tail recursion. But if you have a program that doesn't contain a loop at that point at all, or loops over something different, then you've got a different program.

<comment><USER26> > If you're genuinely mutating the AST, you'll end up with a different program > that does something different or performs differently.

If the original program was written in, say, F# and used a lot of pattern matching, then you could rewrite a _straightforward_ moral equivalent in C# that used lots of visitors. Both the AST and IL will be substantially different, but the performance and functionality will be similar.

<comment><USER25> You're thinking at a much more concrete level than I am. When I think about an AST differ for the purpose of identifying copied code, I'd abstract out different ways of writing loops; I'd unify polymorphic calls to pattern matching; I'd break function call graphs down to a forest of basic blocks. In other words, I'd tune my AST comparison to actually look for algorithm similarity. I wouldn't be so trivially gamed.

Even better, I'd work with a traced execution, and examine isomorphisms between call graphs and data structures.

</comment></comment></comment></comment><comment><USER27> I have do (academic) work [1, 2] on finding semantic code duplication. Two points that I have learned about code duplication:

1\. There is are a lot, A LOT, of code regions that share similar constructions when analyzed in terms of dependencies. Dependencies only consider data flow and control dependencies. Where a statement X is control dependent on another statement Y (usually a if-condition or loop-condition) if Y _decides_ whether X executes.

In my studies I have found modestly sized Java programs (~ 75 KLOC) have > 500 million patterns representing duplication in their dependence graphs.

2\. Not all dependence structures which are "duplicate" would be considered duplicated by a _human programmer_ [2]. It takes discernment by someone familiar with the code base to decide whether or not regions are actually duplicated.

I would argue you can draw similarities using automated metrics between disparate code bases. Those similarities are not evidence of copying. To decide whether similar regions are actually copied you would need to do further and subjective analysis. Without directly evidence of copying it would be very difficult to make a solid claim one way or the other. But, given the vast amount of similar code regions that exist (and assuming most code is not copied) I believe it should be given the benefit of the doubt.

Note: I have not studied density of duplicated code between different projects. The above is merely an conjecture based on my experience.

[1] [http://hackthology.com/rethinking-dependence- clones.html](http://hackthology.com/rethinking-dependence-clones.html) [2] [http://hackthology.com/sampling-code-clones-from-program- dep...](http://hackthology.com/sampling-code-clones-from-program-dependence- graphs-with-graple.html)

</comment><comment><USER28> Having gone through a similar trial, this is painfully familiar. The reality seems to be that the legal process optimises for whoever argues the most manipulatively (using every dirty trick in the book) rather than actually optimising for truth.

Perhaps this is inevitable: rather than hoping to successfully convey a logical scientific argument to the judge/jury/arbitrator about a deeply technical area, instead the lawyers find themselves painting a more subjective simplified story that the audience may relate to... at which point you have to fight like with like. It's almost a post-truth kind of situation; the actual reality is irrelevant; it's instead how well the theatrical posturing is executed or how compelling the simplified version of the story is.

That said, I have no idea whether Carmack is in the right or wrong here. In my instance the system worked in my favour, but I suspect it had a lot more to do with having lawyers who were good at debating than the almost incidental fact that objectively and demonstrably I had done nothing wrong...

<comment><USER29> Absolutely concur. In an arbitration case, our attorney told us beforehand that (pardon the language) "the only bigger whores than lawyers are expert witnesses".

The outcomes seem to depend mainly on, as you say, which side can spin the most compelling simplified story that resonates with the arbitrator/jury.

As another poster says, perhaps use of analogies should be banned (Harry Potter in this case for goodness sake).

</comment></comment><comment><USER30> This is a thorny issue because say you could make the argument that if I spend 5 years writing and rewriting code from 0.1 to 0.8 at company A to understand a domain, then I move to company B and write 0.9 and 1.0 in the same domain and make a pile of $$$, then company A should somehow be compensated even if the code itself is not exactly the same (which it wouldn't be since you learned from mistakes made).

On the other hand if you take, for example, an engineer who learns to build bridges and builds a lot of bridges in one company, and then moves to a different company still building bridges, one would not think that just because they build a great bridge at company B, company A should be compensated.

Where do you draw the line between 'improving your craft / becoming a better software developer' to 'taking a company's IP'?

<comment><USER31> The issue of potential IP transfer is why companies sometimes include non- compete clauses in their employment contracts.

I don't know if they can be enforced in the relevant jurisdictions though.

</comment></comment><comment><USER32> An interesting statement on things that may be relevant in the grand scheme of the trial, but, unless I'm really missing something here, the verdict was about busting an NDA (the Code being downstream of the NDA violation). Hence the $500M vs the $4B in damages. If it was just about code, then I'm sure Carmack's points would hold up just as well as they read in hindsight. But - and it's a big but - keep in mind his post is in a vacuum of sorts by not relating it back to the mechanics of how the case was actually ruled. Again, I might be missing something in his post, but I don't think so.

As for this part:

> _The notion of non-literal copying is probably delicious to many lawyers, > since a sufficient application of abstraction and filtering can show that > just about everything is related. There are certainly some cases where it is > true, such as when you translate a book into another language, but copyright > explicitly does not apply to concepts or algorithms, so you can't abstract > very far from literal copying before comparing. As with many legal > questions, there isn't a bright clear line where you need to stop._

Non-literal copying is why I like the "Gaye Family vs. Blurred Lines" verdict and why a wide swath of business folks and industry creatives hate it. Also, it's nearly a 1:1 if you want to talk about Code like Carmack does or Music like Musicians do. It's a totally reasonable thought experiment for one side to claim "Well, a jury of non-musicians ruled the wrong way and called it derivative" to which I'd counter "Well, a jury of expert musicians would probably rule the same way because they understand the nuances even better than the average lay person." Again, these are hypothetical arguments with real-world consequences, but I don't think these cases can be easily ruled upon. There's always going to be some hard feelings at the end of it.

<comment><USER33> I believe the "non-literal copying" thing simply refers to the insight gained from the R&D time he spent on Occulus while still an employee of Zenimax. 90% of HN members know that this deep understanding is FAR, FAR, FAR more valuable than blindly copying over code.

It was this deep understanding that made Occulus valuable and it was seemingly funded on Zenimax's time and dime.

<comment><USER34> Perhaps it's not an issue, but it's not like Carmack learned stuff as an employee. He has been the principle architect of the graphics engine for every product he's worked on for the last 20 years. How do you take that out of your head when you change companies but stay in the same field?

</comment></comment></comment><comment><USER35> "There are objective measures of code similarity that can be quoted, like the edit distance between abstract syntax trees, but here the expert hand identified the abstract steps that the code fragments were performing, made slides that nobody in the courtroom could actually read, filled with colored boxes outlining the purportedly analogous code in each case. In some cases, the abstractions he came up with were longer than the actual code they were supposed to be abstracting."

This is because of the abstraction filtration comparison test that courts use: [https://en.wikipedia.org/wiki/Abstraction-Filtration- Compari...](https://en.wikipedia.org/wiki/Abstraction-Filtration- Comparison_test)

I think it's garbage too, but such is life.

I don't think you can blame the expert for producing something that goes along with what the current law is.

</comment><comment><USER36> Dave Beazley (Python expert and author of SWIG) was an expert witness and analyzed a huge body of code. This is his fascinating talk about his adventure:

[https://www.youtube.com/watch?v=RZ4Sn-Y7AP8](https://www.youtube.com/watch?v=RZ4Sn-Y7AP8)

So, what happens when you lock a Python programmer in a secret vault containing 1.5 TBytes of C++ source code and no internet connection? Find out as I describe how I used Python as a secret weapon of "discovery" in an epic legal battle.

Slides can be found at: [https://speakerdeck.com/pycon2014](https://speakerdeck.com/pycon2014) and [https://github.com/PyCon/2014-slides](https://github.com/PyCon/2014-slides)

</comment><comment><USER37> >* After he had said he was "Absolutely certain there was non-literal copying" in several cases, I just wanted to shout "You lie!". By the end, after seven cases of "absolutely certain", I was wondering if gangsters had kidnapped his grandchildren and were holding them for ransom.*

This seems to be a very common reaction when reasonable people are put through a lawsuit.

<comment><USER38> At least we can get few bits if information knowing the expert witness was old enough to have grandchildren.

</comment></comment><comment><USER39> Link doesn't work for me, I just get a popup telling me to sign up to Facebook, and when I close it the page is blank?

EDIT: Incognito mode fixed it. [http://pastebin.com/KxAyvqyM](http://pastebin.com/KxAyvqyM)

</comment><comment><USER40> The relevant court decision seems not to be about the topics about which Carmack writes:

[https://arstechnica.com/gaming/2017/02/oculus-execs- liable-f...](https://arstechnica.com/gaming/2017/02/oculus-execs-liable- for-500-million-in-zenimax-vr-trial/)

"While the jury found that Oculus had not misappropriated trade secrets from ZeniMax—a major part of the case—it determined that Oculus co-founder Palmer Luckey had violated a non-disclosure agreement with the company when working on early prototypes of the Rift VR headset"

"Oculus is liable for $300 million in the verdict ($50 million for trademark infringement, $50 million for copyright infringement, and $200 million for breaking the NDA), while Luckey owes $50 million and former Oculus CEO Brendan Iribe owes $150 million (both for false designation). Oculus CTO John Carmack, who previously worked for ZeniMax and was accused of stealing code and destroying evidence, is not personally liable for any damages. ZeniMax had sought damages of $6 billion in the case."

</comment><comment><USER41> To be clear, I (like many of you) have poured countless hours into John Carmack's games, and stand in awe of his work. I also have considerably less capability than most HN users in programming, etc. So I may have a _large_ pro-Carmack bias going into this...

Having said that, the concept of "non-literal copying" is, IMHO, nonsense. And dangerous. So I'll agree with Carmack on that much. And while I don't know (I don't think it's been revealed, although I haven't been following the case closely), it seems that Carmack had some hand in writing the original code, so of course the code will have similarities.

Also, the analogy of "Harry Potter with the names changed" is, from what I can tell, a pretty bad one for what actually happened.

In short, I think this is pretty much nonsense.

OTOH, Zenimax does have so evidence of wrongdoing on Carmack's part, so it's not like their case has no basis, and I am inclined to be biased toward Carmack. And there's always the possibly that I have no idea what I'm talking about and I'm just stark raving mad.

</comment><comment><USER42> I learned long ago (at the beginning of my legal career) that a determined litigant can essentially cook up expert testimony on demand as needed to further some tendentious goal or other.

It is dressed up to be "scientific," "learned," etc. but, when objectively analyzed, it is in reality utter garbage. _But_ , and this is a big "but," it is the sort of garbage that cannot be refuted conclusively owing either to uncertainty in the science involved or uncertainty in the facts and assumptions on which it is based.

By this, I don't mean there are experts who have integrity and who will not allow their good names and reputations to be cynically used for such purposes.

But, for every expert of such integrity, it is (sadly) pretty easy to find others who can be bought.

Don't know what happened in this case but I instinctively can sympathize with the author here that this is what might easily have happened.

</comment><comment><USER43> Well, who was the expert witness?

</comment><comment><USER44> I often wonder if John was involved with the story telling aspect of his games, not just the code.

<comment><USER45> I think it has a recurring theme that any time he got involved, things didn't go well so he decided to let go of it.

That's not to say he didn't have influence; many times the games' stories were derived FROM the tech, not the other way around. Doom was originally supposed to be much more story- and character-driven, but that took a backseat given the focus on the development side was on the gameplay mechanics.

Also, this quote:

> Story in a game is like a story in a porn movie. It's expected to be there, > but it's not that important

From "Masters of Doom".

<comment><USER46> He was the Dungeon Master in their Dungeons & Dragons games. John Romero loved Carmack's campaign world, but John Romero destroyed it, Opening up a gateway to hell, and then immediately regretted it (but they got the idea for Doom from it), since it was a long running campaign in a world completely designed by John Carmack. I think Heretic/Hexen and Quake also borrowed heavily from their D&D games. Masters of Doom is an awesome book.

<comment><USER45> Good point. I forgot about that.

</comment></comment></comment></comment><comment><USER47> My father was an expert witness in electronic discovery cases for many years in the 90s and early 00s.

I remember so many stories growing up about similar situations. The main lesson was that it was actually way easier to be an expert witness when your side was totally in the wrong, because you could just make stuff up. Explaining the truth about how computers work is hard, compared to making up simpler explanations that don't happen to actually reflect reality.

</comment><comment><USER48> Expert witnesses is kinda crazy in this country. One of the most famous cases had to be the father wrongfully executed because of it[0].

Law enforcement are also trained to talk about their years of experience and how they're an expert when giving testimony regardless of facts and juries love it.

[0] [https://en.wikipedia.org/wiki/Cameron_Todd_Willingham](https://en.wikipedia.org/wiki/Cameron_Todd_Willingham)

</comment><comment><USER49> I think the last sentence is the meat of the post.

The biggest credential of some expert witnesses is the number of trials for which they have been an expert witness.

Put your testimony out there for the public and let other experts critique it. Stake your reputation and your career on it.

</comment><comment><USER50> Does this mean if I write code that does X at company A, and then move to company B and am asked to write code that does X, I would be copyright infringing my own code I wrote at company A unless I purposely went out of my way to write it completly differently?

Because chances are I still remember how I did it, or even if I don't, chances are I'll come up with pretty much the same way, as it's inherent to my code style and thought process.

I guess it's something I'll be taking great care in doing if this ever happens to me.

</comment><comment><USER51> Too many people read too much into "peers". These are your legal peers, not social or educational equivalents. These are fellow citizens. So long as they arent royals, elected officials or cops they are peers. (Lawyers can technically serve on juries but every court i know doesnt want them to do so, nor law students.) We live now in a society with fewer official class systems, or we at least now better separate legal classes, and so easily forget the original intention of such words.

</comment><comment><USER52> I recently served on a jury in an assault and battery case. The defense offered a psychological defense and called a psychologist as an expert witness. The witness offered a few theories which were credible but not evidentiary. The prosecution, however, did not attempt to contradict the claims, and instead argued that the witness was too expensive and did not have much contact with the defendant.

It was a mistrial due to jury deadlock, as reasonable doubt was present.

</comment><comment><USER53> John Carmack has a history of betraying employers (e.g. softdisk). Hard to take him seriously when the evidence is he said she said and the law decided against him.

</comment><comment><USER54> Battle between experts trying to convince people from the street is very fragile system. Arguments start from the scratch every time.

The idea of making expert witness testimonies as part of ones academic record is great.

Scientific expert testimony literature should be cumulative in the same way as legal precedents are. Code copying is case that comes up again and again. There should be convergence towards scientifically justified ways for experts to determine copying.

</comment><comment><USER55> I'm still surprised Facebook still has Facebook.com/somescript.php style links unless it's not as simple as it looks in the url. I know there's about 4 layers of loadbalance/cache and layers of that in data centers but any idea about the framework or code design?

<comment><USER56> It would be kind of bad to break something named "permalink", wouldn't it?

</comment></comment><comment><USER57> That's a great analogy that I'm surprised I haven't heard before.

If I write a book called Gary Potter, about a boy who finds out he's a wizard and goes on (specific) adventures with his friends Jon and Germione... I'd be in trouble.

But a book about someone wishing for a better life > finding out they secretly had great powers > struggle to gain control/mastery of the powers > must use the powers to save the world from evil... that's just a story framework and applies to Harry Potter, Star Wars, Hercules(the Disney version at least). There's variations in why they want the better life (abuse, boredom, etc), how they find out, and what the exact evil is. but if you abstract far enough, they're all basically the same story.

Using exact code and changing variables like "main_character_name" is clearly infringement, but using a similar abstracted flow shouldn't be. ESPECIALLY in code. If your end goal is to put pixels on a screen. you run algorithms to determine pixels > add those pixels to a buffer > send the buffer to the screen. There's not many other ways to do it. There's only so many ways to implement some things. And if a way is good, independently coming up with the solution isn't copying. In fact, if I came up with a different solution, it would be sub-optimal. For some problems, the abstracted flow of the code is defined for you. "We're doing VR, we want X, Y, Z to happen"... There's not a whole lot of options on how to do that.

If Oculus only figured out the flow-charts based on copying zenimax's prototype, then they perhaps are copying. And thats the argument zenimax is shooting for. "this is a non-trivial, non-obvious solution they only got by copying us".

But the next question is: at what level of abstraction am i just changing variables at a large scale? If I wrote a book about a young boy living with abusive relatives because his parents died and he finds out he's a wizard and moves away to a magical wizard school, does that count as copyright infringement? There's a point where it is, and a point where it isn't.

I can never find the source, but I once saw a 256x256 picture of yoshi, that immediately below it had a 128x128 version of the same picture (stretched to 256x256). Below that was 64x64, etc until the 1x1 was just a green pixel. Along the side it said "at what point does it stop being copyright infringement? If you started at the top (the high res), you could get pretty far down and still go "oh yeah, this is still the same image". but if you started at the bottom, you could get much higher and go "this is some weird abstract art of random pixels"

</comment><comment><USER58> Should he be talking about this? Don't they still have a chance to appeal?

</comment><comment><USER59> The goal of the expert is not to give a "truth", it's to convince the jury. That's a very different job.

</comment><comment><USER60> Isn't the testimony already a matter of public record?

<comment><USER61> He mentions that it's not.

"... his expert testimony in trial is under seal, rather than in the public record."

</comment></comment><comment><USER62> All laws relating to Intellectual Property need to be abolished.

</comment><comment><USER63> Loser's lament.

</comment><comment><USER64> Carmack is a hero for me but he seems to come across as very naive here. I employ other people now and employees who argue this sort of stuff isn't copying are toxic to the culture and ethics of work. When you are paid a salary every month, I actually don't care if you are productive. But the least I expect you to be is loyal. And I am no monster for expecting that.

<comment><USER65> Loyalty in a free market is a pipe dream. Extremely few employers will stay loyal to their employees when times get rough, say when an employee gets very sick and can't work for several years. Thus no rational person should have any long term loyality to an employer if the market providers better alternatives.

<comment><USER64> This is a big reason why a US company might prefer immigrant workers. Local workers tend to treat their jobs more as transactions and less like a relationship with their employer. Immigrant workers come with built in motivation to stick to their employers and stay loyal.

</comment></comment></comment><comment><USER66> "The expert witness circuit is surely tempting for many academics, since a distinguished expert can get paid $600+ an hour to prepare a weighty report that supports a lawyer's case. I don't have any issue with that, but testifying in court as an expert should be as much a part of your permanent public record as the journal papers you publish. In many cases, the consequences are significant. There should be a danger to your reputation if you are imprudent."

So basically he's kind of threatening the expert witness now?

<comment><USER67> If the "expert witness" is either lying _(in court)_ or incompetent, shouldn't there be consequences to their professional reputation? Or do you think such things should be consequence-free?

<comment><USER68> There is already a consequence to lying as a witness, it's called perjury prosecution.

It's unfortunately infrequently pursued when the perjury occurs in a civil case, but if you want to agitate for consequences, agitate for prosecutors willing to enforce the law.

(Or for a civil cause of action for perjury, but that would have to be carefully crafted to prevent an infinite regress of lawsuits.)

<comment><USER69> > There is already a consequence to lying as a witness, it's called perjury > prosecution.

Except that this is an " _expert witness_ ", and not a regular witness. The expert witness offers his/her opinions.

<comment><USER68> A statement about one's own opinion can be materially false, and provably (in the sense the word applies in the legal system) so.

Certainly, an attorney can (and this is true of any witness, not just expert witness) coordinate with a witness on a line of questioning which carefully avoids any questions where perjury would be necessary to avoid damaging the case. That's why we have witnesses subject to examinstion by opposing counsel, as well.</comment></comment></comment></comment></comment>


College Student's Speech Makes Him King of the Nerds - <AUTHOR> http://mashable.com/2013/08/21/college-student-speech-georgia-tech/

<comment><USER2> I'd ve said the guy was a computer scientist, surprisingly he is mech eng!</comment>


The Xi-Editor - <AUTHOR> https://google.github.io/xi-editor/index.htm <comment><USER2> link is wrong should be [https://google.github.io/xi- editor/index.html](https://google.github.io/xi-editor/index.html)

<comment><AUTHOR> tnx, i resubmitted it.</comment></comment>


Harvester pulls record amount of drinking water out of thin air - <AUTHOR> https://newatlas.com/materials/harvester-drinking-water-thin-air/ <comment><USER2> It would most likely be far more cost-effective to use nothing more complex than good-quality air-conditioner.</comment>


Ask HN: Mesos vs. OpenStack - <AUTHOR>

This is a question of who will win the private cloud war? mesos+docker or openstack? love to hear more thoughts. <comment><USER2> This is kind of the wrong question. You can trivially run openstack ontop of mesos and vice versa.

Mesos is bigger on services, openstack is bigger on virtual machines. They are different solutions for overlapping, but fundamentally different problems.

Here is an example of Ebay running mesos ontop of openstack:

[http://www.ebaytechblog.com/2014/04/04/delivering-ebays- ci-s...](http://www.ebaytechblog.com/2014/04/04/delivering-ebays-ci-solution- with-apache-mesos-part-i/#.U9qEHnVdWEJ)

<comment><AUTHOR> Why not mesos+docker running on all HWs, which eliminates the point of running openstack at all. Google has been doing this for yrs.

<comment><USER3> Google has been running docker for years? I kind of doubt that ;)

<comment><AUTHOR> borg+container, they are not fan of hypervisor, thus guess they are not fan of openstack as well.

<comment><USER4> borg+container == mesos+docker+kubernetes-on-mesos

<comment><AUTHOR> well, the point is that why you need openstack sincere the fullstack is there like you suggested?</comment></comment></comment></comment></comment></comment>


How Post-Watergate Liberals Killed Their Populist Soul - <AUTHOR> http://www.theatlantic.com/politics/archive/2016/10/how-democrats-killed-their-populist-soul/504710/?single_page=true <comment><USER2> I find it really odd that this article dodges the advent of the Democrat Superdelegate system after the Hunt Commission.</comment>


Mindfulness, Stress Reduction & Healing; Jon Kabat-Zinn, 2007, Google Tech Talk - <AUTHOR> http://video.google.com/videoplay?docid=-7128459048009318384#

<comment><USER2> Clicked, and the video says "Video is currently not available." Anyone else having this problem?

Anyways, here is the YouTube link: <http://www.youtube.com/watch?v=rSU8ftmmhmw></comment>


Facebook: Who Pressed The Big Red Button? - <AUTHOR> http://thenextweb.com/facebook/2010/12/16/facebook-who-just-pressed-the-big-red-button/

<comment><USER2> Memories was a feature that was desperately needed. Everything else is meh. How many times do they have to change the design of the same pages without adding anything new?

<comment><USER3> What's with the name? 'Memories'.

<comment><USER4> I think it's great. It's better than "Status Archive" or something similar. And really, it _is_ your memories... going back to 2004 and seeing what I was up to then is certainly going to trigger some nostalgia for me.

</comment></comment></comment><comment><USER5> I don't understand the fan page changes. If they don't have any tabs, what becomes of apps that run on fan page tabs?

</comment><comment><USER6> They put a Quora in my Facebook!

</comment><comment><USER7> Or somebody is going to be fired or this is a marketing rumor-apple like movement...

</comment><comment><USER8> What a terrible "accident"! Really though, Memories is a pretty damn good idea. If it will auto organize photos and content based on semantics and date, that would be very cool.</comment>