Worst Bug Ever - <AUTHOR> http://swanson.github.com/blog/2013/01/20/worst-bug-ever.html

<comment><USER2> IMO Matlab is the worst programming language widely used in production.

This perfectly fits my expectations of Matlab:

\- i is a mutable, but should be constant

\- It has a horrible, non obvious name because mathematicians are used to it. (And they are not used to var names longer than two chars)

\- It is in the global scope.

\- It's a feature that has a special syntax (sort of).

/edit Apologies for php programmers, maybe php is worse. But I'm not sure.

<comment><USER3> > \- i is a mutable, but should be constant

It's not really mutable, because it isn't a variable. However, if there is a variable by that name, Matlab will use it instead of the complex constant.

> \- It a feature that has a special syntax (sort of).

The problem is rather _a lack_ of special syntax (or too much leniency). Python also has built-in complex numbers:

        >>> 1 + 1j
        (1+1j)

but it doesn't allow you to omit the constant before the j:

        >>> 1 + j
        Traceback (most recent call last):
          File "<stdin>", line 1, in <module>
        NameError: name 'j' is not defined

You can do it the same way in Matlab to avoid this problem (1 + 1i is always OK since implicit multiplication is not allowed), so the problem is that Matlab lets you use i syntactically like a variable when it in fact isn't.

<comment><USER4> It will only give that NameError if 'j' is in fact undefined; 'j' can be used as a variable which would lead to a similar bug in Python (and why is it 'j' instead of 'i' ... they're both common one-letter variables).

<comment><USER5> Electrical engineers use j instead of i to distinguish it from current (that is commonly referenced by i).

<comment><USER6> Whereas physicists use capital I for current. (And J for the current density.)

<comment><USER2> I have seen j for current density only so far. But physics are convinced you can tell the difference by context (which is usually true, but probably impossible for a compiler, even if physics is "typed" with units). For handwritten text at least my professors managed to come up with at least two different ways to write each letter to avoid confusion. My (not serious) attempts to use Chinese characters haven't been very successful.

</comment></comment></comment></comment></comment></comment><comment><USER7> I was using a very well-known popular computation software direct competitor to ANSYS Multiphysics. I was using it since early version 3 days, through 4.x over a period of 3 years in my phd in the field of fluid-structure interaction (e.g. fish in flow). The computations usually took over several days on many cpus for 1 second of real-life simulation, making testing quite painful. Hence, many studies do not involve testing like in software field.

I was calculating a parameter enhancement where my calculations were showing a great 180% enhancement; this result would make it into very high ranking (15+ IF) paper right away. By then, several people had written numerous papers on this matter using this multiphysics software in well-known journals. But my gut feeling didn't let me. Eventhough my professors were pressing me to publish, I took 6 painful months (toward the end of my phd) to rigorously test the software itself, having nothing to do with my work, since it was closed- source.

After 6 months, I found out that a velocity term used in their stabilization algorithm does not use relative velocity, but absolute velocity, leading to enormous amount of fake diffusion into the model. I had to travel to another city in europe to convince them that there is such bug.

Every single work in my field that used this software till then had been severely invalid. And what the developers did: they fixed the bug in the next sub-release without putting it in the changelog like nothing happened.

<comment><USER8> I don't suppose you were able to publish a peer-reviewed journal article calling previous results into question, were you?

</comment></comment><comment><USER9> I know the feeling, but I think I have a bug that beats even this one...

Once, I was working on the "alias" keyword in MacRuby, and getting it to work with methods defined in Obj-C. Since MacRuby objects are just extensions on Obj-C objects, some methods are defined in Obj-C and require special casing in order to alias them to a different identifier. For example, if you want to alias "Array#length" to a different name, you'd actually be aliasing "-[NSArray length]".

So I do everything I should need to do to make it work: get a handle to the object, the object's class, the method defined for the class, a selector for the new name...but no matter what I tried, every time I aliased the method, and attempted to call the method by the new name, I would always get the same error: "Method not defined".

Finally, I decided to dive deep with gdb and find out exactly what was going on. To my shock and amazement, I came to realize that I _was_ properly aliasing the method but, due to the nature of NSArray's implementation as a class cluster (and the fact that Obj-C doesn't have proper virtual methods like C++), the implementation of "-[NSArray length]" was literally:

        - (NSInteger)length
        {
          [NSException raise:@"Method not defined." format:@"%"];
        }

<comment><USER10> Are you saying that when a name is aliased in MacRuby, the alias becomes a copy of the IMP from that particular class, such that subclass overrides of that method are never invoked? Because that sounds like a bad idea.

<comment><USER9> Aliasing in MacRuby (or Obj-C for that matter) involves adding a new entry to a class's method table with the new name but a pointer to the old implementation. Method lookup from subclasses to superclasses proceeds the same as it would for any method.

<comment><USER10> Yes, but if you message the subclass with the alias, only the superclass's method gets invoked, because the alias apparently contains the IMP of the superclass's method. Or it must, if what you said before is to be believed.

This is pretty nasty, because it breaks the idea of message sends and subclass overrides of methods. There's two ways you could solve it:

1\. Set the IMP to a static function that looks up the method it should forward to and re-send the message there. This is hard to do in a manner that supports all arguments, and you'd need at least 2, possibly 3 versions anyway (one for regular, one for strret, and on some architectures, one for fpret). It would probably need to be written in assembly in order to forward the arguments.

2\. More recently (in iOS 4.3, and whatever the corresponding OS X release was), the runtime got a nice little function called imp_implementationWithBlock(). This takes a block and returns an IMP. Bill Bumgarner has a nice blog post about it ([http://www.friday.com/bbum/2011/03/17/ios-4-3-imp_implementa...](http://www.friday.com/bbum/2011/03/17/ios-4-3-imp_implementationwithblock/)). You could use this function on every alias in order to invoke the correct method, passing along the correct arguments.

<comment><USER9> That may be nasty, but that's how the semantics of method aliasing in Ruby work. Try this Ruby snippet to see what I mean:

        class Foo
          def say; puts "hello"; end
          alias shout say
        end

        class Bar < Foo
          def say; puts "howdy"; end
        end

        Foo.new.say #=> "hello"
        Bar.new.say #=> "howdy"
        Foo.new.shout #=> "hello"
        Bar.new.shout #=> "hello"

<comment><USER10> Oh ugh. When is an alias not an alias? When it's Ruby, I guess.

</comment></comment></comment></comment></comment></comment><comment><USER11> I had a similarly fun time back in the 90s debugging a VBA script a co-worker had written for an Excel spreadsheet. She was looping through a bunch of time values...

... so she named her variable "time"

... which, when modified, would (naturally) set the system time of the computer.

I'm not sure whether they've fixed that one yet.

<comment><USER12> At least once a quarter I'd have a student working on some programming assignment on one of our unix-ish systems where they named the executable "test" and couldn't figure out why it wasn't doing anything they expected.

I also got to the point with FORTRAN (yes, I'm old) students where I'd cut them off and tell them to count the columns and make sure each line started with 6 blanks, not 5.

My favorite bug is still some piece of C code buried in a library that was like this:

       if (flag == TRUE) ...

Or there's the old for loop typo (much easier to detect on today's high-DPI screens):

       for(float x=0.0; x < 3,4; x = x + 0.1) ...

<comment><USER13> I don't get what's wrong with the flag check apart from being a bit too verbose.

<comment><USER14> TRUE is going to be 1. The flag value might have the value 0x2, 0x4, 0xsomething_else, etc., which will not equal exactly 1 (TRUE).

What you want varies but is probably something like:

        if (flag & FLAG_VALUE) ...
        if (flag) ....
        if ((flag & FLAG_VALUE) == TRUE) ...

that kind of thing.

<comment><USER15>
        if ((flag & FLAG_VALUE) == TRUE) ...

That would still check if flag is 1, right?

<comment><USER16> That is correct.

<comment><USER14> Indeed, great catch (this is why I use things like QFlags now, as it has a simple "testFlag" method)

</comment></comment></comment></comment></comment></comment></comment><comment><USER17> > _the imaginary constant i in MATLAB was getting overwritten!_

The constness of _i_ in Matlab is also imaginary, which is pretty terrible.

<comment><USER18> Yeah, I'm hoping by "overwritten" he really meant "shadowed". I don't know MATLAB, but being able to change the value of i seems both useless and dangerous.

<comment><USER17> I think it's shadowed globally, which has the same effect as it being overwritten. But I'm not certain.

</comment></comment></comment><comment><USER19> Unlike many of the other commenters, I don't see much problem with having "i" mean the imaginary unit. Where MATLAB falls down though is:

1.) not having lexical scoping

2.) not having warnings when a global is overridden in a local scope

these are pretty inexcusable. For anyone looking for a "better MATLAB", Python has NumPy, SciPy and Matplotlib which are pretty much everything.

<comment><USER20> >> _For anyone looking for a "better MATLAB", Python has NumPy, SciPy and Matplotlib which are pretty much everything._

I hear this at work all the time, "just use Python, we're going to use Python everywhere we use Matlab from now on". It simply doesn't work like that.

Usually it's programmers who are too overly enthusiastic about replacing Matlab with Python, because they are comfortable with all aspects of programming languages. The people typically writing the most complex Matlab code aren't, more often than not they are physicists or mathematicians. Python + SciPy + Matplotlib is not nearly as accessible as Matlab, and you drag in all kinds of typical 'software engineering' problems that don't exist when you stay inside the Matlab IDE and its related tools. I can conjure up a plot in Matlab and manipulate it incrementally for example, while Matplotlib often requires a lot of crazy data mangling and setup code that isn't required in Matlab (trust me, I've spent months re-implementing a library of Matlab plotting scripts in Matplotlib, and the code of some of them is completely untraceable to the original Matlab code, and full of stuff completely unrelated to plotting). Also, lots of Matlab code relies on all the crazy implicit operations and type juggling allowed by Matlab, many of which are almost fundamentally incompatible with a well-defined programming language suited for production code.

This is all even ignoring the fact that Matlab comes with a pretty extensive library of toolboxes that have no equivalent in SciPy or Numpy. If you're really determined and know how to program in Python, you can basically get a lot of stuff done without requiring Matlab, but if you've always been using Matlab like a prototyping tool or a very elaborate graphical calculator (and this constitutes a very extensive part of Matlab users), Python + all its scientific libraries is no substitute.

<comment><USER19> Good points, and that provides some perspective I hadn't considered.

However, I still think my points about where MATLAB falls down are valid. There is no reason for it not to have lexical scoping, or warnings about when you are doing stupid things. It just doesn't have them, and it causes frustrations like the one in this article.

<comment><USER20> Oh totally agree, Matlab-the-language is terrible, and Python is a million times nicer to work with. My comment was only about the suitability of Pyhton + Numpy + SciPy + Matplotlib as a Matlab replacement for non-programmers.

</comment></comment></comment></comment><comment><USER21> I made a similar error once working in Adobe Air (if I recall):

        for (x = 0; x < thing.length; x++) { }

I didn't put a "var" before the x- so Air used the window.x object, and shifted the entire contents of the window to the right by a seemingly random number of pixels.

To look at it like that, it's blindingly obvious. But it was in the middle of a whole lot of other code, and of course, the for loop ran perfectly fine.

</comment><comment><USER22> Nice anecdote, but that's a rather uber-sensational title - not in a good way.

<comment><USER23> Agreed; I expected it to be the story of Therac-25 (<http://en.wikipedia.org/wiki/Therac-25>)

 _It was involved in at least six accidents between 1985 and 1987, in which patients were given massive overdoses of radiation, approximately 100 times the intended dose. These accidents highlighted the dangers of software control of safety-critical systems, and they have become a standard case study in health informatics and software engineering._

<comment><AUTHOR> It's linked to in the first paragraph.

</comment></comment></comment><comment><USER24> Thats not so much a "nice bug" but rather a big fat warning sign for anyone trying to use MATLAB.

</comment><comment><USER25> Matlab: the language designed so poorly it makes PHP look good.

<comment><USER26> I love that you said that, I didn't read my daily dose of crying about PHP today. Thanks!

</comment></comment><comment><USER27> This is not the worst bug ever. Nice story, however, the title is way over the top.

<comment><USER28> Yeah, I was thinking more like Therac-25 giving people radiation overdoses or the Patriot missile failures due to floating point roundoff error

</comment></comment><comment><USER29> Kind of along the same lines in the sense of being _my_ biggest facepalm moment; many years ago in my first professional programming gig we had full SA access to our sql server. We regularly did troubleshooting via direct SQL statements (a practice I've since shied away from). One particular day, we were troubleshooting a problem in our monthly billing process. I went to delete a few rows and foolishly forgot to include a limiting statement on the WHERE clause, which resulted in most of the data being blown away.

Thankfully, we were able to restore that table from a backup and rerun everything to rebuild the table's state to a correct place ... but suffice to say the experience taught me many things about attention to detail, and how debugging in production should happen.

Good times!

<comment><USER30> A good practice there is to do tricky DELETE/UPDATE inside a transaction. In e.g. Postgres, executing DELETE/UDPATE tells you number of affected rows, so if you see 1243 rows were updated rather than 1 as you expected, you can ROLLBACK.

<comment><USER31> Another good approach is to replace 'delete' with 'select count(1)' or something similar. The where clause can remain the same, you know how many rows will be affected and there's less overhead of remembering to set up a transaction, writing an update statement, rolling back, etc.

<comment><USER29> Yeah, if I ever did have to troubleshoot in SQL directly, I definitely started doing this ... only changing it to update or delete when I was 100% sure it was affecting only what I wanted to affect.

</comment></comment></comment></comment><comment><USER32> As a mechE, I was brought up on Matlab. We were taught to use ii, jj, kk... as our index variable names. Many years later, I still prefer them to i, j, and k.

<comment><USER33> Like stuttering of a shell-shocked veteran, it's a battle scar of Matlab survivor.

</comment></comment><comment><USER34> That is a pretty awful MATLAB perk, using 'i' as the sqrt(-1) (and indexing the arrays from 1...).

It makes polyglot programmers make a lot of silly mistakes. It would have been SO easy to call the imaginary constant just 'I' instead of 'i'...

<comment><USER35> Or just make it not mutable. Throw a big, fat error when someone assigns value to i.

<comment><USER34> This would have been the best way to learn it, some kind of 'CRASH Eh! MAN your files have been formatted and all this project has been deleted because

i=sqrt(-1)

Sorry for the mess, hope you learn from this...'

But I keep tripping over the same stone...

<comment><USER8> Presumably the parent meant to throw an exception and show a script-level stack trace, not silently exit with no diagnostic messages.

Insisting on doing something, even if it's the wrong thing, is much worse in most applications than refusing to run in order to avoid producing incorrect and misleading results.

</comment></comment></comment></comment><comment><USER36> Just want to say thanks to all the Hyperbole Police in this thread pointing out this isn't _actually_ the worst bug ever.

<comment><AUTHOR> Ya - I was going for a reference to Comic Book Guy: <http://en.wikipedia.org/wiki/Comic_Book_Guy>

Thought the periods would make it clear. Oh well!

EDIT: Looks like the periods got removed from the title - I can see why that would make the reference a bit more obtuse.

</comment></comment><comment><USER37> it's the most frequent error you can do when writing 'a-little-bit-more' complex scripts using MATLAB. it's sad to say but MATLAB takes too much care of backward code compatibility and they didn't fix this! all matlab's newcomers do that error. :)

</comment><comment><USER15> On my first programming job (2004), I was adding shopping cart functionality to a proprietary CMS written in PHP. For some reason I couldn't get a simple item listing to work. I had spent several hours debugging it before I found the problem.

The CMS had a database query result class with a method named something like "numberOfRecords". It returned the number of records minus one...

I told my boss about this, and his answer was "Yeah I know, but we don't want to fix it because a lot of code depend on that bug."

</comment><comment><USER38> I'm baffled by the fact that the single most common name for an integer variable has a special meaning (though it's not a reserved word). I'm pretty sure this is a bug in MatLab, not in OP's code.

But still I was wondering: isn't that code horribly un-idiomatic for an array based programming language?

<comment><USER39> I'd bet i has been used to denote the imaginary unit long before it was used as a loop counter in programming languages. Wikipedia appears to date the former to sometime in the 18th century.

</comment></comment><comment><USER40> Oh the joys of side-effects. Wouldn't you love to have a programming language that doesn't have them?

<comment><USER41> It would never work for something as side-effect prone as pure mathematics!

</comment></comment><comment><USER42> If that is the worst bug ever, I would die a happy programmer. Btw, why would anyone use the 3*i form when there is the much clearer 3i form?

<comment><USER43> What if you're dealing in imaginary units that aren't constant? E.g. foo*i.

<comment><USER42> Ah, right. I am surprised there is not a better syntax for it though.

<comment><USER44> It should still be ' foo * 1j '

</comment></comment></comment></comment><comment> why-el Don't trust it too much: <http://cm.bell-labs.com/who/ken/trust.html>

</comment><comment><USER33> Fun, but there are worse things than scope shadowing bugs. At least this one is deterministic and can be zeroed in with a few trace prints.

</comment><comment><USER45> Python (before version 3) leaks variables from list comprehension, while it seems natural to assume that the comprehension has its own scope. Makes for some funny bugs:

      i = ...
      ... # lots of code
      [ do_stuff(i) for i in range(n) ]
      ...
      # whoops, our outside i has now been modified

</comment><comment><USER46> If you call this the worst bug ever (t = [-infty, +infty]), then what's this?

[https://github.com/MrMEEE/bumblebee-Old-and- abbandoned/commi...](https://github.com/MrMEEE/bumblebee-Old-and- abbandoned/commit/a047be85247755cdbe0acce6f1dafc8beb84f2ac)

</comment><comment><USER47> That is not what I call "worst" bug. It is a pretty cool bug. Screw Matlab for hiding this from you.

</comment><comment><USER48> Maybe change the title.

</comment><comment><USER49> I spent a few hours debugging an OpenGL program I was working on only to realize that I forgot to account for normalized coordinates. All I had to do was individually multiply x and y by the canvas size.

Always start with the small, obvious stuff first!

</comment><comment><USER50> For some reason, I've found more bugs like this in MATLAB than in any other domain. Ever.

</comment><comment><USER51> Colleague had a long long debugging session doing MDA based code generation, turns out JET (java/ibm/eclipse) template engine had a stupid arithmetic bug, the kind you'll only investigate after everything in the stack has been checked.

</comment><comment><USER52> Thanks for posting this -- I just went and fixed just such a loop in my own code.

<comment><AUTHOR> Ha! Glad I could help :)

<comment><USER44> Me too! I changed my array counter to 'j'. :P

I work in matlab a lot and have had this error creep up several times. It's very annoying, to say the least.

<comment><AUTHOR> The better fix - which I mentioned as well - is to use `2+3i` instead of `2+3*i`. The former will always use sqrt(-1).

<comment><USER44> I agree. I think I picked up this syntax from Python. My response was intended as tongue-in-cheek answer since Matlab allows both i and j as the complex number. I mainly see this when editing or modifying scripts written by others in my lab and have to pinpoint their counter usage.

</comment></comment></comment></comment></comment><comment><USER53> Why did he get the correct results when the script was run stand-alone?

<comment><USER54> He was passing the numbers directly when running stand-alone. His other code was looping through an array.

</comment></comment><comment><USER55> I don't know what I would do without list comprehensions. I can hardly imagine using iteration loops and counters anymore, and this is one among many reasons why.

<comment><USER56> List comprehensions don't really handle performing a list of actions for their side effects alone.

<comment><USER55> The example in the code was not a side-effect. They were creating a new list by invoking a function on every member of the old list.

Here's the same thing re-written in Python:

      d = [compute_diffraction_at_wavelength(x, WAVELENGTH) for x in LensLayers]

Or oldschool Python (and still faster, I think):

      d = map(compute_diffraction_at_wavelength(WAVELENGTH), LensLayers)

</comment></comment></comment><comment><USER57> Oh, the things a little syntax highlighting can prevent...

</comment><comment><USER58> "those where"

</grammar nazi>

<comment><AUTHOR> Fixed, thanks. [https://github.com/swanson/swanson.github.com/commit/81c14ed...](https://github.com/swanson/swanson.github.com/commit/81c14ed32a5d603bff95ccac48f5e405902c4514)

</comment></comment><comment><USER59> Further evidence that j is the one true imaginary unit.

Title is still sensationalist.

</comment><comment><USER60> Who upvoted this?

</comment><comment><USER61> Nice article but there's a lesson here.

First I'll come with an old bug of mine (early nineties). The bug was very hard to reproduce because it was very very occasional (but terrible in consequences). I simply couldn't find it. I knew I was probably smashing some memory somewhere (I was not that experienced in C) but couldn't find where.

So what did I do after weeks of hunting the bug? I moved my design away to an entirely deterministic one and started recording all the inputs the program received and then I could replay them. At one point, logically, the bug happened. And so was it "recorded". Because my program was now deterministic, I could simply feed it the inputs (and the time at which they happened) and, surely enough, the bug was there.

And then finding the cause of the bug was of course trivial.

What's the lesson?

Well, first obviously bugs that you cannot easily reproduce because you cannot "reproduce the state" are typically kinda hard to track down.

The less "state" in your program, the easier it is to reason about your program and the easier it is to reproduce the state.

The lesson?

Functional programming rocks.</comment>


Secure file storage, Storing and searching files…. Securely? - <AUTHOR> https://cloudblogs.microsoft.com/microsoftsecure/2018/10/16/secure-file-storage/?WT.mc_id=MSFT365-Hackernews-tajanca <comment><USER2> 'Securely' as in 'completely visible to Microsoft'. This is an unavoidable issue with data and operations on them being 'in the cloud'.</comment>


Intel wants to charge $50 to unlock stuff your CPU can already do (2010) - <AUTHOR> http://www.engadget.com/2010/09/18/intel-wants-to-charge-50-to-unlock-stuff-your-cpu-can-already-d/ <comment><USER2> I assume this is being posted because something similar has been found on Haswell, or whatever.

Complaining about this is just not knowing what is going on. Developing a processor is a very complicated matter, and it's financially impossible to develop several models for all the possible different performances. So they just design and build the best possible device (with exceptions of course).

Now, I want a not-so-powerful device. Why should I pay the price of all the R&D it took to squeeze the top performance? Maybe I just won't buy it because, hey, it's too expensive.

On the other side, I want the top of the line. And Intel spent lots of money developing it, in practice raising the price for everyone else. Shouldn't I be made to pay more to compensate for that?

At the end of the day is a matter of creating artificial tiers so different needs pay more or less of a share of the R&D and production cost. Something similar happens when paying $100 for an upgrade from a 16GB to 32GB iPod. The NAND Flash chip costs about $5, but I'm paying the premium of the extra investment the company did, the additional supply chains, etc.

Of course this just is invisible to the customers most of the time, and this 'upgrade' cards make it obvious now. Anyone familiar with testing equipment (oscilloscopes, etc...) will know that is a very common marketing technique.

<comment><USER3> The problem with Intel's tiers is that you can't even buy the top of the line device: the concept doesn't exist due to the artificial market segmentation.

For Haswell, you can buy the 3.4Ghz one with TSX (4770), or the 3.5Ghz one without TSX (4770K).

You cannot have the faster speed and TSX. Why? No reason. Well, someone at Intel figured that people that want to have fast multithreaded software must be business users that don't care about clockspeed. Or something. Don't ask me, I think it's the most bonehead retarded thing ever.

<comment><USER2> Maybe it's just that in that case the silicon won't support it, we don't really know how the architecture is. It's also possible that it heats too much with both things. Who knows.

Indeed it would be stupid not to offer something, just because. Charge $2000 for it and someone will buy it.

<comment><USER4> _Maybe it's just that in that case the silicon won't support it, we don't really know how the architecture is._

Indeed it is conceivable that transactional extensions wouldn't work when the clock multiplier is unlocked, but I can't yet imagine why.

</comment></comment></comment></comment><comment><USER5> Functionally, it seems like the logical combination of binning and the market pressure for cheaper machines. (Binning-> <http://en.wikipedia.org/wiki/Product_binning>)

Say you are Intel and Gateway/Dell/OEM orders X lower-end CPUs... great. However, you don't have enough that binned that far down. So, you have to cripple some ones that are more capable. Do you write it off? No, offer a way to have the end-user pay for the difference! Hey user, you want to unlock the difference between what OEM paid for and what we happened to give them? Here's a painless way to unlock it.

Functionally most of the electronics you buy are capable of more. Take a look at this: [http://hackaday.com/2013/03/18/hack-removes-firmware- crippli...](http://hackaday.com/2013/03/18/hack-removes-firmware-crippling- from-nvidia-graphics-card/) Okay, so what happens? Either everything's great and you got a free upgrade or the thing has a bad shader or memory chip and now you have an unreliable card.

I don't agree with the whole up-sale aspect of it, but, it just strikes me as Intel trying to make back some money on what likely was a crap deal for them.

<comment><USER4> _Functionally most of the electronics you buy are capable of more. Take a look at this:[http://hackaday.com/2013/03/18/hack-removes-firmware- crippli...](http://hackaday.com/2013/03/18/hack-removes-firmware-crippli..). Okay, so what happens? Either everything's great and you got a free upgrade or the thing has a bad shader or memory chip and now you have an unreliable card._

Another thing you might see in a substantially similar hardware product line is that resistors indicate the hardware model, but the lesser models are missing essential components that are present in the higher models. Sure, the firmware will think it's running on the higher model if you swap out the resistors, but it'll promptly crash when it can't find that DAC and EEPROM that are supposed to be there.

</comment></comment><comment><USER6> This is also something that's often seen in oscilloscopes. For example, many Agilent scopes can be upgraded by buying a key.

The impact for users is twofold. On the one hand users that don't upgrade still have to pay for a few (unused) components. On the other hand, if users decide they want to upgrade later on, they don't have to throw away/sell their 800$ scope or send it back to the factory. You can buy the basic model when you have the money and upgrade later.

It might be a bit awkward to buy something you pretty much own already, but it's still the most rational option.

If only humans were more rational...

<comment><USER7> When Tektronix aged my 'scope out of support, the last free firmware update unlocked all those extra cost firmware-locked upgrades. A real class act.

I wonder if Intel will do the same a few years down the road?

</comment></comment><comment><USER8> In raging over this practise, there's an unstated assumption: That the alternative is selling the full, unlocked chip at the consumer price point.

This is very unlikely to be the case. Like first class on a flight, the pricing of the high end is likely subsidising the lower end. Of course, there's still a profit margin on the low end offering, just not enough that it can stand on its own.

So the alternative to this practise is a cheaper high-end chip for everyone, it's more expensive (or same price) high-end chips, which means the low end of the market get's priced out and have to make do with last years model.

<comment><USER9> Intel is not going to sell you any chip at a loss these days... Intel makes money... they profit... loss-leaders don't give them that.

</comment></comment><comment><USER10> This is a weird concept and I'm trying to figure out how they intend to market it. It seems to me that people buying cpus look for the best performance at the right price, build the machine, and then 3 or 4 years later start the process over again. They won't unlock the cpu right then and there unless it fits within the price range, and they wont do it years later when they're looking to upgrade their rig because they'll haven't forgotten or there will be something much much better.

<comment><USER11> IBM has been doing this for Mainframes and pSeries/RISC boxes for a long time. Other mainframes did this as well.

On the pSeries boxes, you buy a base level of CPU cores, and you can buy licenses to unlock additional cores for a period of time. It gives you cloud- ish type features (pay for what you need, when you need it) with on-premise hardware.

It was a really common licensing model when "data processing" was all about batch workloads. Now that we're building virtualized scale-out systems versus client/server systems with fixed resource requirements, it's a model that can make sense again. If you think about it, EC2 is another way of accomplishing the same thing.

</comment></comment><comment><USER12> The 386SX and 386DX were exactly the same silicon, but the cheaper model had a fuse deliberately blown to disable the maths coprocessor. It all comes down to whether you take the view that you're buying a piece of silicon, or a slice of R&D.

<comment><USER13> No, that was the 486SX/DX. No 386s had built-in FPUs - the 386SX was a DX with a 16-bit data bus and a 24-bit address bus.

<comment><USER12> Bitrot in my brain... :)

<comment><USER14> I was _certain_ , but I still looked it up ;-). There's nothing like making a mistake on Hacker News to show one how to eat humble pie.

</comment></comment></comment></comment><comment><USER15> So did this ever blow over or are these upgradable chips still on the market? Did anyone ever hack the upgrade mechanism?

</comment><comment><USER16> This is known in the economics literature as product crimping. Perhaps surprisingly, this can make everyone better off (not only Intel but also the purchasers of both the crimped and uncrimped goods). See "Damaged Goods", by Deneckere and McAfee [1].

[1] [http://scholar.google.com/scholar_url?hl=en&q=http://www...](http://scholar.google.com/scholar_url?hl=en&q=http://www.econ.iastate.edu/faculty/langinier/teaching2005/615-fall2005/paper- hom2/damagegoods.pdf&sa=X&scisig=AAGBfm2DuFTO-d5ZF1R3RB3s1CNYNYp1VA&oi=scholarr)

<comment><USER17> The only thing I can see proof of here is that Intel is a monopoly, as they wouldn't be able to get away with this if they were actually competing against each other.

<comment><USER16> Did you read the paper? They cited a case where crimping occurred in printers even though IBM and HP were competing. Also, ARM and AMD are producing chips, even if they don't target the same subset of the market.

Regardless, the point is that by crimping Intel may be making themselves and their customers better off. Sure, we could posit that with an even more competitive chip market we'd be even better off, but that doesn't negate the fact that we might be worse off than the status quo if crimping were disallowed given the current competitive landscape.

</comment></comment></comment><comment><USER18> Everyone wants in on DLC, turns out... as much as we all complain about it, we still hungrily buy it.

It's greedy, and douchey, but as long as they don't bring back the Celeron I'm OK with them trying this.

It isn't the case that your average user today can tell the difference between a computer that's brand new, or that's 4 years old. Chips aren't the bottleneck anyway.

</comment><comment><USER19> Many software vendors do very similar things: by entering a different registration key (that costs more money) you can unlock features of the existing software that were previously unavailable to you.

For example, by entering a registration key, you can upgrade "Windows 7 Professional" to "Windows 7 Ultimate" and get more features. It only takes a few moments to do, so it's clear that little or no additional software is being installed on your machine.

</comment><comment><USER20> Man AMD had better pull up its socks & provide some competition or there will be much more of this nonsense in future.

</comment><comment><USER17> I don't really mind this, as all it means is that I have to download a program of some torrent and -viola- better performance.

</comment><comment><USER21> It's failed model. If the hardware is there people will find a way to unlock it without paying. Haven't license key proven that by now?

</comment><comment><USER22> Look if you guys are even goign to argue this is a good thign , ive already given up the war on DLC has been lost and it was won by the stupid.</comment>


Monster Energy Drink Cited in Deaths - <AUTHOR> http://www.nytimes.com/2012/10/23/business/fda-receives-death-reports-citing-monster-energy-a-high-caffeine-drink.html?_r=0

<comment><USER2> The problem with putting labels on cans saying the amount of caffeine (and often other ingredients) is that just listing the measurement doesn't put it into context of how much that is.

I had to look it up, and apparently coffee as either a brewed cup, or espresso shot has between 100-200 mg of caffeine. So the amount of caffeine in the can isn't out of range for what would be expected of a caffeinated energy drink.

However, we don't require that coffee have a label showing how much caffeine is in it either.</comment>


Non-blocking transactional atomicity - <AUTHOR> http://www.bailis.org/blog/non-blocking-transactional-atomicity/

<comment><USER2> This is very cool. Simple enough to reason the invariants easily. I guess one of the key insights is that each data has a canonical server owner which enforces the consistency of the writes of the data at a single place.

I have one question regarding the determination of the latest version of a set of peer data when overlapping transactions occurred.

Suppose initially s1/x=0 at server s1 and s2/y=0 at server s2. Client1 updates s1/x=10 and s2/y=10 at transaction v1, and at the same time client2 updates s1/x=20 and s2/y=20 at v2. Suppose the clients contact the servers in different order and the update messages arrive at the servers in reverse order, such that s1's pending queue is [x.v1=10, x.v2=20] and s2's pending queue is [y.v1=20, y.v2=10].

After client1 and client2 send the make-good command, s1's good queue is [x.v1=10, x.v2=20] and s2's good queue is [y.v2=20, y.v1=10]. When a third client3 tries to read the latest value of x or y, what is the latest value of its peer data? It looks like depending which data client3 starts with, it would get a different version of the peer data? Like starts with s1/x got s1/x.v2 == 20 and s2/y.v2 == 20, while starts with s2/y.v1 == 10 and s1/x.v1 == 10.

Am I missing something or this is the semantic in determining the latest values of peer data?

<comment><USER3> (edit: post author, [not OP]) here. Thanks for the feedback!

> I guess one of the key insights is that each data has a canonical server > owner which enforces the consistency of the writes of the data at a single > place.

Well, this is the way I presented it, because it's easiest to understand. But, if you want a replicated system that provides HA, there are alternatives ([http://www.bailis.org/blog/non-blocking-transactional- atomic...](http://www.bailis.org/blog/non-blocking-transactional- atomicity/#ha-note)).

> When a third client3 tries to read the latest value of x or y, what is the > latest value of its peer data? It looks like depending which data client3 > starts with, it would get a different version of the peer data? ... Am I > missing something or this is the semantic in determining the latest values > of peer data?

Good question! This ultimately comes down to how you want to handle concurrent writes. Many distributed databases use a "last writer wins" strategy when reconciling concurrent updates (that is, the correct behavior is specified to be that the database serves the highest-timestamped version of a given data item). Now, in your example, the clients both started their writes at the same (real-world) time and used this time as a basis for their timestamp, so the "last" write is undefined. We need a way to break the "last writer wins" tie. In practice, this can be something like the client ID appended to the last few bits of the timestamp or even a hash of the value written--as long as the tie- breaker is deterministic (that is, different replicas don't decide different "winners"), it doesn't really matter which is chosen.

In practice, to avoid storing every value ever written, you'd want to provide some kind of "merge" function for multiple writes, and, in our implementation and in often practice, this is last writer wins (plus some deterministic tie- breaker for identically timestamped but distinct writes).

<comment><USER4> It took me time but I understood why the suggested race condition with two or more clients can't happen. But this require that pending information is not "overwritten" and preserved until becoming good. The x=10 is then never overwritten by the x=20 and the meta data allows to distinguish the two possible value of x. If a third client gets a good x=10, it will get the y value with the same meta data and which is 10, not 20 or 0.

The only problem that bogs me a bit in this algorithm is that transaction time sequence is not guarantied and preserved. Some clients may see x=y=10 and others x=y=20 during the transaction period. This happen if one client starts writing x=y=10 with s1 and the other y=x=20 with s2. Clients requesting x and y starting with s1 will get x=y=10 and those requesting x and y starting with s2 will get x=y=20. When the transactions completes, the values finally stored in x and y as good may be 10 or 20.

So the algorithm ensures consistency, which is the most important and useful property, but with many concurrent write transactions, the ending DB content may be a bit somehow unpredictable.

This can be a problem when one needs to do operations like x-=1 and y-=1 on the database as for seats reservation in a plane for a travel with two flights for instance. How would this be done ?

<comment><USER3> Good questions.

In the example configuration in the post (i.e., two servers with no replication), during the period between the write start and end of phase two of writes, reads can return either value written (x=y=0 or x=y=1). If we want to enforce the property that once one read returns the second write, all subsequent reads (that begin after this read) will return the second write (or a later write), which is called linearizability (<http://cs.brown.edu/~mph/HerlihyW90/p463-herlihy.pdf>), then whenever servers serve from 'pending', they should move the writes to 'good'. This is safe because, if a client reads from 'pending', it means that the write must be in 'good' elsewhere and therefore is stable.

As for how to order the writes, real-time clocks can provide a fairly useful timestamp mechanism. Databases like Cassandra use this real-time ordering for timestamping, which appears to work well in practice. Alternatively, you could use a distributed sequence number generator to totally order transactions. But real-time should be fine.

As you point out, this atomicity property doesn't address (all) application- level integrity constraints; it doesn't handle _isolation_ between transactions. As I discussed in the post and in another comment (<https://news.ycombinator.com/item?id=5784030>), if your application-level integrity constraints are such that your updates are not commutative (that is, concurrent updates should not be allowed), then you'll need to block in order to guarantee database integrity is not violated. This is separate from atomicity, but it _is_ important to remember. In your example above, two writers might both simultaneously reserve the last seat on a plane unless they synchronize.

Effectively, with non-commutative updates, you need greater isolation than is provided by the algorithm in the post (which effectively provides Read Committed isolation). Achieving greater isolation is possible but you'll lose the non-blocking property (again, due to the requirement to avoid concurrent updates via higher isolation like serializability rather than due to atomicity). But, for many applications like 2i and the multi-puts I mentioned at Facebook and Twitter, updates are commutative.

</comment></comment></comment></comment><comment><USER5> Very cool and simple. The notion of "time" may be a bit misleading but actually the client just requires to generate an unguessable rand, think at it like getting a string from /dev/urandom, so actually time is completely not part of the protocol which is great (also avoiding to deal with client IDs can be nice in practice).

<comment><USER3> Yep--thanks! I've updated the post accordingly :)

</comment></comment><comment><USER6> The algorithm does seem nicer than a 2PC protocol in that there is no need for a 2PC coordinator. By distributing the metadata as it does the clients and servers can figure out what has been committed and what hasn't. However it doesn't appear to directly include semantics for aborting transactions which is a pretty important part of a distributed transaction protocol.

The paper admits the algorithm does not guarantee termination but I would have liked to see more details on the failure scenarios regardless (minor details in footnote 3). It's not clear to me what writers see (if anything) when a write fails.

The paper does talk about how non-overlapping transactions won't block each other (which is nice but not a solution) and how one could add the ability to abort and trigger a cleanup by the use of a good (user supplied) failure detection module. But having a reliable node failure monitor that can react fast enough to ensure availability is really the hard part.

Would love to see more on aborting transactions next.

<comment><USER3> > However it doesn't appear to directly include semantics for aborting > transactions which is a pretty important part of a distributed transaction > protocol.

Yep, I left this out to avoid confusion at first. There are some details in the "What just happened?", but the basic idea is that any aborted write will be stuck in "pending." Same for failed writes; writers won't see these. The algorithm presented actually guarantees "Read Committed" ACID isolation.

> But having a reliable node failure monitor that can react fast enough to > ensure availability is really the hard part.

Well, you'll remain available for reads and writes, but the size of "pending" might grow. You essentially need asynchronous distributed garbage collection, which will stall in the presence of partitions and may require the failure detectors I mentioned.

> The paper does talk about how non-overlapping transactions won't block each > other (which is nice but not a solution)

I don't see how this isn't a solution for transactions that desire last- writer-wins semantics. If, as in the examples I listed, writes commute, then a blocked write shouldn't stall others. If you want to prevent Lost Update or Write Skew anomalies (i.e., concurrent update), then you'll have to give up availability and/or block.

<comment><USER7> > I don't see how this isn't a solution for transactions that desire last- > writer-wins semantics. If, as in the examples I listed, writes commute, then > a blocked write shouldn't stall others.

Doesn't writes that commute mean that there was no contention to begin with? Ideally, if I have a balance of $100 in my account and try to spend $60 in two different transactions, one should come back as failed before the purchases are complete.

> If you want to prevent Lost Update or Write Skew anomalies (i.e., concurrent > update), then you'll have to give up availability and/or block.

There is a difference between giving up read and write availability. My ideal database should be read-available at all times, but guarantee that writes are atomic and durable (and give up availability for this guarantee).

On the whole, this looks pretty neat. I like the idea of the client being responsible for the writes being committed on the servers. The client is then free to choose how to implement the IO, but ultimately, if a single client experiences a failure and a single write doesn't go through, it is usually a better outcome than a write going through and then replication between two servers breaking.

What are your thoughts on quorum-based voting in distributed systems? E.g.: your protocol but with the requirement that a write is considered stable if only most (vs all) of the servers involved have it marked as "good".

<comment><USER3> Good questions!

> Doesn't writes that commute mean that there was no contention to begin with? > Ideally, if I have a balance of $100 in my account and try to spend $60 in > two different transactions, one should come back as failed before the > purchases are complete.

Whether or not conflicting writes are a problem or not depends on the application semantics. For example, if I'm, just adding items to a set, then my updates commute. But if I have a constraint that says that elements in the set need to be unique, then my updates don't (logically) commute any more. Ultimately, this is application-specific. The "CALM Principle" (<http://www.bloom-lang.net/calm/>, <http://vimeo.com/album/2258285/video/53904989>) captures this notion of "logical monotonicity" resulting in safe operation despite concurrency. Most of the applications I mentioned, like 2i updates and the social graph example, commute.

For non-commutative operations (and there are plenty), you'll need a stronger model like serializability or sequential consistency that necessarily blocks to prevent concurrent update (or otherwise aborts concurrent updates).

> My ideal database should be read-available at all times, but guarantee that > writes are atomic and durable (and give up availability for this guarantee).

This is definitely one point in the spectrum; the question is whether you want to give up availability and write performance. But if you look at workloads like those in the Spanner paper, this is reasonable for many applications.

> What are your thoughts on quorum-based voting in distributed systems? E.g.: > your protocol but with the requirement that a write is considered stable if > only most (vs all) of the servers involved have it marked as "good".

There's a difference between quorums over _replicas_ over the same data item and quorums over different data items. Using quorums over replicas would help ensure that the replicas provided properties like linearizability or register semantics like Dynamo or other systems that provide an option for "strong consistency." But it's not clear to me that quorums over replicas for different data items would provide the same atomicity property--does that make sense?

I intentionally left many of the issues of replication to the footnotes in the post--mostly for readability and clarity--but I believe the technique is applicable to both linearizable/"CP" and "eventually consistent"/"HA"/"AP" systems.

<comment><USER7> > For non-commutative operations (and there are plenty), you'll need a > stronger model like serializability or sequential consistency that > necessarily blocks to prevent concurrent update (or otherwise aborts > concurrent updates).

The problem to me seems to be that the interesting problems always come across the case where updates are not commutative. For example, real-time sensor reading where frequent updates completely override previous state are very easy to get right if you don't need the latest data at all times. You simply get recent or slightly stale data, and the service stops responding if all sensor data is old. There are many solutions to this case, and some are less complex than your solution.

However, when dealing with a read-update-write type transaction where the values you write depend on the values you read will indeed require stronger guarantees. Here is where a lot of systems get into trouble. They seem to either implement the fuck it mode or attempt to do some kind of distributed locking which usually takes a huge performance hit even if the network is fine.

> This is definitely one point in the spectrum; the question is whether you > want to give up availability and write performance. But if you look at > workloads like those in the Spanner paper, this is reasonable for many > applications.

Yes, the idea is that the system becomes read-only but the data remains consistent and online. In most workloads that I've seen this is the desired behavior. For some reason I haven't seen this behavior implemented yet, though it's possible I just haven't looked at the right data store.

> There's a difference between quorums over replicas over the same data item > and quorums over different data items. Using quorums over replicas would > help ensure that the replicas provided properties like linearizability or > register semantics like Dynamo or other systems that provide an option for > "strong consistency." But it's not clear to me that quorums over replicas > for different data items would provide the same atomicity property--does > that make sense?

Unfortunately, you lost me here. I am thinking of the quorum over each register as being the pass-fail for whether a transaction can go on. Basically, client A connects to servers X and Y and says "set t = 1; set s = 2; ... commit;". During the ..., client B connects to server Y and Z and says "set u = 9; set t = 3;". Here, client B should fail since client A has not committed. This is determined by the fact that the majority of the servers (Y and Z) cannot agree that t is available for writing. In this case, client B will receive a "success" from Z and a "fail" from Y, which will prompt it to roll back the transaction and start over.

In other words, move the responsibility to coordinate a successful write from the datastore nodes to the single point: the client.

<comment><USER3> Thanks for the feedback--these are great points. (post author, btw)

> The problem to me seems to be that the interesting problems always come > across the case where updates are not commutative.

I agree that many application-level integrity constraints can't be satisfied with commutative updates. However, I've been surprised how frequently they can work for many web applications and how frequently "fuck-it" integrity constraint maintenance is employed given i.) faster database operation and 2.) asynchronous compensation mechanisms (e.g., bank overdraw fees) in the event of constraint violation. But your point is well-taken!

> Here, client B should fail since client A has not committed.

I think I understand your example, though I'm not clear as to whether s and t are stored on both X and Y or separately as s on X and t on Y. But, in general, writes in 'pending' should not block the insertion of other writes into 'pending' (in 2PC parlance, prepared but non-committed transactions should _not_ block other transactions from preparing). This is fundamental to the non-blocking property of the algorithm here (i.e., [http://www.bailis.org/blog/non-blocking-transactional- atomic...](http://www.bailis.org/blog/non-blocking-transactional- atomicity/#so_what_just_happened)). So if client A hasn't committed, it shouldn't stop client B from committing. If client A and client B's writes don't commute, then the clients should use a stronger protocol/isolation level than the effective Read Committed that the NBTA algorithm here provides (doable but requires blocking). Does that make sense?

<comment><USER7> X, Y, and Z in this example would all have complete copies of the data, so _s_ and _t_ are stored on all three nodes. Client B shouldn't be blocked, but rather told that an error occurred and that client B is trying to modify data that is in flight. In other words writes may fail, but they fail atomically, and in such a way that the client may retry without fear of leaving data in an inconsistent state.

I suppose this method is somewhat similar in philosophy to Software Transactional Memory, but is distributed. The hard part here seems to be things like replication and garbage collection. What if client A fails before it commits? How long should the cluster lock the values of _s_ and _t_ before client B is able to modify them? I suppose some type of heartbeat from client to server would be good. Since there would be a strong guarantee that if a transaction fails, just retry it and nothing bad happens, then client A can come back online and try again once connectivity is restored.

</comment></comment></comment></comment></comment></comment></comment><comment><USER8> This model is somewhat similar to how Datomic models transactions -- Datomic uses immutable datoms and includes a transaction ID with each datom: "A datom consists of an entity, attribute, value and transaction (time)" (<http://www.datomic.com/rationale.html>).

However, Datomic's design is based on a single, centralized transactor that pushes out all the new transaction information to an index that is distributed to all its clients (peers), whereas NBTA would enable distributed transactors.

See "The Datomic Data Model" <http://www.datomic.com/rationale.html#DataModel>

From Datomic's FAQ <http://www.datomic.com/faq.html> ...

How does Datomic provide ACID guarantees?

The transactor serializes all transactions, and each transaction runs against a stable view of the database, and succeed or fail in their entirety. Transactions are not acknowledged until after they are logged in storage. Atomic read/modify/write operations can be performed by database functions, which run on the transactor within the transaction. Note that Datomic can provide ACID guarantees without utilizing read-transactions, nor read locks, due to the presentation to the query engine(s) of the database as an immutable value.

\---//---

Rich Hickey on Datomic's design...

Intro to Datomic (12 min) <http://www.youtube.com/watch?v=RKcqYZZ9RDY#>!

The Design of Datomic (60 min) <http://www.infoq.com/presentations/The-Design- of-Datomic>

The Datomic Architecture and Data Model (46 min) <https://vimeo.com/45136212>

<comment><USER3> Very cool, thanks! In general, immutable data items makes NBTA much easier; there's only one element in either 'good' or 'pending' for each data item at a time. The benefit of a single centralized transactor/ID generator is that it provides a simple total ordering on updates. The downside is that updates must (often) complete sequentially to guarantee safety and that the centralized service can become a bottleneck/challenge during failures. That said, I'm a big fan of the Datomic vision and look forward to seeing what they're able to accomplish.

</comment></comment><comment><USER9> If you clear metadata, the following pathological case results in an inconsistent read:

* x = 0 and y = 0, starting out in the "good" state with metadata cleared;

* client A reads x with no metadata, then for whatever reason blocks or is delayed;

* client B writes x = 1 and y = 1, completes the transaction, and the metadata is cleared;

* client A reads y = 1.

This shouldn't be an issue unless you aggressively clear metadata and have a long-running client.

<comment><USER7> In that case, if client A wants to do something with the value x, it should instruct the server to place metadata on it, essentially saying that it's reading it with the intent to update. This could be implemented as "update x with value of get(x)". That way it would be placed into pending with metadata.

Having a heartbeat between the client and the server could also help prevent the write from getting purged from "pending" prematurely.

<comment><USER3> I'll also add that the metadata requirements aren't huge--typically 8 bytes for the timestamp and N*(bytes per key) for the keys. In our implementation and the benchmarks that we provided, we don't garbage collect the metadata as it's small and isn't a serious overhead.

</comment></comment></comment><comment><USER10> In the mode where a client is responsible to move writes from 'pending' to 'good', it wasn't clear to me what happens if the client dies before contacting all servers in the second pass?

Does the data remain stable, or must some additional work be performed to correct the inconsistent state?

<comment><USER3> > Does the data remain stable, or must some additional work be performed to > correct the inconsistent state?

If you want client writes that reached all servers to become visible, then the servers will have to perform the move from 'pending' to 'good' on their own (by communicating asynchronously). The notification of write stability is idempotent, so it doesn't hurt if both clients and servers perform this notification.

FWIW, in our implementation, servers perform the second step instead of clients (which can be made more efficient via batching).

</comment></comment><comment><USER11> I think there's a typo under "good, pending and Invariants"? Isn't the caption supposed to be "not okay / (x=1 should be in pending or good)"

<comment><USER3> Post author here. Hmm. The invariant we're trying to maintain is that any write in 'good' should have its transactional "siblings" in either 'good' or 'pending' on their respective servers. So if we are trying to write x=1 and y=1, then, if x=1 is in the x server's 'good', then y=1 should be in the y server's 'good' or 'pending'. But in the example under "not okay", y is not present in either.

<comment><USER11> Ok, then I just did not understand that part yet.

</comment></comment></comment><comment><USER12> In theory this idea looks OK. I'd be interested if someone implemented it on Riak or Cassandra? What are the conclusions?

</comment><comment><USER13> Getting only 4.8% percent of throughput for reads doesn't seem stellar. Why is this so bad?

<comment><USER3> Ha--good catch! I don't know if anyone else noticed that. I meant to say "within 33-4.8% of the peak throughput..." That is, we're 95.2% of the peak, not 4.8% of the peak. Fixed, thanks!</comment></comment>


"There's a Slide For That" - Generic Slides to Avoid in your Pitch - <AUTHOR> http://navfund.com/blog/5-slides-we-see-in-every-pitch-deck-or-%E2%80%9Cthere%E2%80%99s-a-slide-for-that%E2%80%A6-%E2%80%9D

<comment><USER2> Most VCs have ADHD - they see familiar slide formats and become irritated and impatient. This is also professional disease of many mid-level managers. Several years ago Intel was able to lay off 10000 middle managers to make decision making process shorter.

Not once I had investors looking for 1 millisecond on the slide and telling "Next". Only 5% of the stuff is drawn or written on the slide, I can talk several minutes about it, but VC understand everything from the 1st millisecond ;)

I like people who can ask questions about the contents of the presentation and not only about it's format.

One of the reasons of choosing familiar slide formats (like for example "Competition Quadrant") is that it's familiar, so you can go straight to the content without loosing time to explain the format itself.

</comment><comment><USER3> tl;dr: please make sure you understand and can speak to each of the slides in your deck, especially these most common ones.

Note that this is very different from the takeaway suggested by the title -- avoid these slides. That may be the poster's opinion, but there is no indication in the article that it is that of the article's author.

<comment><USER4> The opinion of the article's author is summarized by ths paragraph:

 _Which started me thinking about the 5 Powerpoint slides we see in every pitch deck. Sure, the slides all look different – different colors, bar charts, pie charts, line graphs. But, basically, they tell the same story. So if and when you use these slides, use them at your own risk – because we've seen them before!_

Incidentally a great book to teach people how to create useful pictures (both for clarifying your thinking and for presenting) is _The Back of the Napkin_. Be warned that it it helps you communicate your message effectively, but it does nothing to help your message be more believable.

The book contains an ironic example of this, by telling the story of a fictional company named SAX Inc. He does a very good job of communicating his thinking about this company, its problems, and what it should do. However my knowledge of software development tells me that the plan he came up with has major pitfalls he is not aware of.

</comment></comment><comment><USER5> Those 5 slides have to cover about 90% of slides in 90% of (business) presentations.

<comment><USER6> 90% of the people giving 90% of business presentations don't know enough of the details or just plain don't have the chops to actually go into detail or back up the assertions they're making.

Utilizing some sort of data-driven process to establish that, "these are our customers priorities" and "we are superior in these priorities" would be black magic to them.

Probably Google's greatest strength is their emphasis on data/empiricism.

<comment><USER7> I think that is also a weakness at Google. They are great at tracking and processing data, but I don't think they are good at intersecting what that data means with the human priorities behind the data. In other words, they can tell if A > B for existing work, but they don't seem to do very good at telling whether C will be better than A, instead choosing to shotgun and hope enough stuff sticks.

Good product management (which early founders must do) is about determining what data to gather by figuring out why the data is important to the future of the product. It costs money/time to gather and analyze data, so knowing you are gathering the right data is important. In the process, you hope to be able to answer whether C will be greater than A.

<comment><USER6> >"Being awesome is better than merely being good"

Why yes, yes it is.

Like yourself, I like to mention how great BMWs are when a couple people are discussing deals on used cars.

</comment></comment></comment></comment><comment><USER8> So what would a VC think of someone who came in with no deck, and just sketched on a whiteboard or paper pad on an easel? The best talks I've been to have been presented that way, with no technology at all. Of course those have been more academic in nature.

</comment><comment><USER9> well, all the deck goes to the dev/null.

it was crafted reading other advices like this.

well... i think i'll do everything reflected/filpped vice versa upside down on the slides. just that a VC could not say 'oh i saw that before 1000 times just with other names'. will catch their attention, make them think, use a system of mirrors just to read!</comment>


The World According to Mad Magazine - <AUTHOR> https://www.nytimes.com/2019/07/12/opinion/mad-magazine-last-issue.html <comment><USER2> an article about the demise of Mad from a publication that was regularly in its crosshairs. nicely ironic.

here's some alternative coverage: [https://reason.com/2019/07/04/mad-magazine- is-dead/](https://reason.com/2019/07/04/mad-magazine-is-dead/)

<comment><USER3> which part is ironic?

nytimes is bound to continue, it's a massive cog in the US propaganda engine, and has always been many many times larger than MAD ever was.

MAD didn't really ever poke fun at NYTimes for being on it's last legs, if anything it poked fun at how disconnected from the real world it and it's authors sometimes seemed to be -- this was made very clear when checking out MADs humor from 89-01 regarding the NYTimes and their coverage of the numerous wars in the middle east.

Nothing about this outcome seems contrary to what anyone thought would happen; MAD has been struggling for years.

</comment></comment><comment><USER4> Related discussion:

[https://news.ycombinator.com/item?id=20351524](https://news.ycombinator.com/item?id=20351524)

And a tribute to Al Jaffee, who has been with MAD for 54 years and at the age of 98 still does every fold-in:

[https://www.vanityfair.com/culture/2017/03/al-jaffee-the- gen...](https://www.vanityfair.com/culture/2017/03/al-jaffee-the-genius- behind-mad-magazine-fold-ins)

</comment><comment><USER5> I love the CD collection, and hope there's some plan to make a new version with the remaining issues.

MAD is better than any history book in that it gives a view to how people really felt about events at the time. It also records minutia in its jokes that otherwise become forgotten after decade go by.

In general it's good to read old humor if only to see how slowly things change. The Realist has their archive online and it's a good read because much of it is still relevant from 1958: [http://www.ep.tc/realist/](http://www.ep.tc/realist/)

</comment><comment><USER6> Is there is some effort to keep MAD going? I learn everything I know from this magazine.</comment>


Amazon CEO says he wants to work more with Pentagon - <AUTHOR> https://uk.reuters.com/article/uk-usa-pentagon-amazon/amazon-ceo-says-wants-to-work-more-with-pentagon-idUKKBN1YB0JN <comment><USER2> > Tech companies have faced challenges when trying to work with the Pentagon. > Last year a defense program, named Project Maven, set off a revolt here > inside Google

The word _here_ made me think it was an employee writing about Google. Turned out to be just a hyperlink. typo? or is this wording common practice

</comment><comment><USER3> I too like money

</comment><comment><USER4> "Socialism for the rich. Capitalism for everyone else"</comment>


ICANN Pushes Ahead With New TLDs - <AUTHOR> http://allthingsd.com/20120111/grab-your-cash-and-warm-up-the-wagon-icann-domain-rush-kicks-off-tomorrow/

<comment><USER2> I don't get why articles keep referring to "22" existing TLDs when there's a huge existing list[1]. Sure a lot of those are restricted, but we've seen a large number of them (.fm .me etc) move beyond their original country-specific intention and be used similar to the ones in the "generic TLD" list.

As for the new ones, I'm more curious than anything, as there are two huge unknowns here. First, people/companies have to buy the domains. Second, the public at large has to accept them. We've already seen TLDs both generally catch on and generally fail (anyone remember the dot-mobi craze?). I'm not convinced that companies are going to have to spend lots of money to squat "mycompany.apple" or "mycompany.hotels" or whatever just to "protect their brand." I mean, companies don't generally worry (AFAIK) about other people's subdomains now, and if new TLDs catch on people may start to view the 'middle' name in the same way. Or maybe not. It's just a big unknown right now.

[1] [http://en.wikipedia.org/wiki/List_of_Internet_top- level_doma...](http://en.wikipedia.org/wiki/List_of_Internet_top- level_domains)

</comment><comment><USER3> I'm super pleased to see this move ahead. Competition has been absent from this segment of the market for far too long. I'm really looking forward to seeing some flowers bloom.

<comment><USER4> You realize that the gTLDs they're selling are not going to work reliably?

Single label names always resolve first on the local domain. Except when they don't. This isn't currently a problem because no admin in their right mind would name their servers "com", "org", or "net". It's a tacit convention that's now being broken. I can think of a few security problems this creates, lord knows how many we haven't thought of yet.

ICANN is selling names in a namespace they don't control.

As Dan Geer puts it, "ICANN has devolved into nothing but a protection racket." <http://geer.tinho.net/ieee/ieee.sp.geer.1107b.pdf>

<comment><USER3> And on that basis, it makes sense to create a monopoly and concentrate very real control over vast tracts of the namespace with unregulated private corporations?

Sounds like a great plan.

I'm always amazed at the large numbers of people that criticize ICANN's work and the very small number that contribute to it.

<comment><USER4> Oh look, the current system isn't ideal either.

 _vast tracts of the namespace_

The thing about (even) the current TLD system is that there is a variety of namespaces and each one by itself is effectively unlimited. Sure, everybody wants to have a .com but when it comes down to it, it really doesn't matter except for brand name purposes. People who need to publish and read information can use something else (that most "rogue" of websites Wikileaks proved this).

On the other hand, moving all the expensive commercial brand stuff to the top level (as ICANN aims to do) really _does_ create a single tract of namespace under monopoly control.

<comment><USER3> yes, and my point unless the market allows for competition to .com, then we've effectively given control of the namespace to one or two organizations - namely Verisign and ICANN.

I don't understand your point about competition creating a monopoly unless you are referring to ICANN"s "powers". It is an artifact of the community. Show up and participate in the monopoly if you are concerned about how it wields its influence and directs its policy.

</comment></comment></comment></comment></comment><comment><USER5> Well at least now I have a good reason to let all of my "land grab" .com and .net domains expire. Also, they are really dumb.</comment>


"Death Dance" Stars Found - May Help Prove Einstein Right - <AUTHOR> http://news.nationalgeographic.com/news/2011/07/110719-white-dwarfs-supernovae-gravity-space-science/?source=link_fb20110720deathdancestars

<comment><USER2> oh for goodness sake. journalists - do you have no better ideas than "prove einstein right"?

einstein was right. his work was an improvement on what went before. we _know_ this. it has been confirmed by many, many experiments and observations. and reported countless times with the same tedious headline.

please, use something a little more original.

<comment><USER3> Well, of course we _expect_ the gravity waves to be there, given the other effects of relativity that have been confirmed so far.

However, scientists still observe, verify, measure. As one other poster mentioned, the equipment to detect gravity waves from afar has not yet been able to do the job (due to opportunity or sensitivity, don't know).

If for some reason they do not find the waves (and the instruments are shown to be good), then there will be some more deep thinking to do: were we really looking at objects different than what we thought? Is there some theory more twisted than relativity that needs to be made to explain the previous results as a special case and also explaining the new results???

<comment><USER2> sure. i have already said the same below.

i am not criticising the physicists, or the person who originally posted the article, or denying how interesting it would be if waves were not found, or disparaging the technical difficulties involved.

i am just tired of the same hackneyed phrase being used in every article related to general relativity. google has over 12 _million_ hits for the phrase "einstein was right". the national geographic is a quality publication. it should know better.

</comment></comment></comment><comment><USER4> I'm trying to figure out how these are different from the binary pulsar PSR B1913+16 (other than they're white dwarfs and the binary pulsar is two neutron stars). PSR B1913+16 is two neutron stars orbiting each other in a tight spiral, and their orbital decay corresponds with what Einstein's general relativity predicts. In fact, the discoverers of PSR B1913+16 (Hulse and Taylor) received the Nobel Prize in Physics in 1993 for showing that this was the case.

</comment><comment><USER5> I'm unclear about how they are spiraling towards each other at 370 mi/s, just 1/3 the Earth-Moon distance apart, yet will take 900,000 years to merge. Can anyone clarify that?

<comment><USER6> They aren't moving directly toward each other at that rate; they are orbiting each other at that rate, and very gradually moving closer.

</comment></comment><comment><USER7> Does anyone know how they would go about observing the gravitational waves? Through lensing effects? Or are they just depending on the orbital decay to demonstrate something?

<comment><USER8> I don't have any expertise in astrophysics (just physics), but I am pretty sure that white dwarfs aren't nearly strong enough to produce observable lensing. You need black holes or whole galaxies for that. Further, these stars are separated by less than the earth-moon distance, so they aren't individually resolvable. The fact that it is a binary system has been inferred from periodicities in it's brightness and spectrum. ("Astronomers measured the stars' relative motions by looking at the light signatures, or spectra, coming from the stars as they eclipse each other.")

I think your guess is right that all the data about the gravitational waves will be taken from the orbital decay. This is actually a decent amount of data since the rate of decay isn't constant, and because the eclipse maneuver isn't a smooth process.

</comment></comment><comment><USER9> Android devs have known about this for ages. That's why there's already a field for it in the sensor manager SDK docs... [http://developer.android.com/reference/android/hardware/Sens...](http://developer.android.com/reference/android/hardware/SensorManager.html#GRAVITY_DEATH_STAR_I)</comment>


How we will survive without newspapers - <AUTHOR> http://reason.com/archives/2009/10/22/fading-print

<comment><USER2> > _Newspapers, their passionate boosters maintain, are a kind of civic > Pilates, the institution that keeps the core of our democratic republic as > tight and toned as Megan Fox's midsection. TV news can show us war zones, > and radio news may offer us aerial assessments of traffic jams, but > newspaper news does the real heavy lifting of journalism. It sorts fact from > rumor. It shines light on government murk and corporate malfeasance and > helps our citizenry make informed and rational decisions. Without healthy > newspapers, we're just Myanmar with better department stores and less > stylish military uniforms._

How strawmanish. It might even be possibly true if only newspapers resorted to journalism.

</comment><comment><USER3> _"Newspapers never systematically reviewed school-teachers, for example, and now they've been scooped by the angry third-grade muckrakers who post at RateMyTeacher.com."_

Wait, what? I had to stop reading here. Someone else please finish and let me know if this was satire or not. If the answer is "not," and this guy honestly thinks "RateMyTeacher.com" is a positive indicator of the future health of journalism in America... well, I'll be glad I didn't waste my time with the rest of this article.</comment>