Slate.js: Customizeable rich text library inspired by Draft.js - <AUTHOR> https://docs.slatejs.org/ <comment><USER2> Ooh, this looks very promising. Nice work.

I'm using draft.js in production (at [http://www.guilded.gg](http://www.guilded.gg)). I just started looking for a replacement for a few reasons:

1\. Draft is pretty broken on Android, despite being marketed as a cross- platform project. The team says that they're not working on fixing it, and it's not trivial to fix. This is a dealbreaker on its own.

2\. I've had users encounter bugs where the Draft editor will serialize state that it can't read - so you can save something, and then cannot reload it. It appears to happen when doing some weird combination of adding and deleting block elements. This is a dangerous and frustrating bug.

3\. In my opinion, the DraftJS API is difficult to use and extend.

Do you (@author) plan to support Android and iOS? If so, I'll definitely give this a shot.

<comment><USER3> Have a look at ProseMirror ([http://prosemirror.net](http://prosemirror.net)). It works quite well on Android and iOS and the maintainer is super responsive and has already fixed a lot of Android/iOS specific issues.

Edit: The landing page of the project does not seem to be optimized well for mobile. When on mobile, make sure to try the editor on [http://prosemirror.net/examples/basic](http://prosemirror.net/examples/basic) instead, as the landing page causes some unwanted scrolling for me on mobile.

<comment><USER4> In what way is Draft broken on Android?

Edit: I see now. I've had a look around and this open pr seems to fix it for draftjs:

[https://github.com/facebook/draft- js/pull/1013](https://github.com/facebook/draft-js/pull/1013)

How are you currently handling Android? We're launching with draftjs in two weeks and need to support Android.

<comment><USER5> On Android there is a high chance your text won't appear, or the keyboard will disappear. Trying to delete text will cause the cursor to jump around if you hold delete down instead of one character at a time. Very frustrating.

<comment> 52-6F-62 The cursor and atomic block deletion issues occur on desktop as well. I filed an issue recently on this very thing.

It seems to be a continuation of the issues with the editor state and DOM falling out of sync.

I can't share the implementation project, but this is the issue: [https://github.com/facebook/draft- js/issues/1089](https://github.com/facebook/draft-js/issues/1089) And the component: [https://robert-fairley.github.io/draftjs-litho- editor/](https://robert-fairley.github.io/draftjs-litho-editor/)

(my component was never meant for mobile use, though. Built for a desktop web app.)

</comment></comment></comment></comment></comment><comment><USER6> I am using Slate for a few months now. Before that we used Quilljs and looked also at other editors before switching to Slate.

Slate helps me so much to speed up development. I don't want even to think about working on my wiki app without using Slate. Sure it is still in beta. Nevertheless it is incredible how fast ianstormtaylor reacts to bugs. I haven't found an issue with Slate that couldn't get fixed within a few days.

Great work! Thx so much ianstormtaylor!

</comment><comment><USER7> Slate is really fantastic.

I tried it out after becoming frustrated with Draft.js, and it's been a joy to work with. Slate's documentation is excellent, the code is well structured and easy to read, and I love how painless it is to implement custom node types and serializers.

</comment><comment><USER8> Seems fairly similar to Daft.js including one of the issues that stopped me from using Draft.js - lack of ability to stylize root Nodes specifically inability to add "text-align" style to paragraphs. Sure, there are some workarounds for that, but those are just terrible hacks bolted on top of the inflexible model system. I had to switch to ProseMirror where that was easier to implement.

Another thing that irritated me a lot about Draft.js is the fact that Entities can't be applied to the same character in content block. One more reason I had to switch to ProseMirror.

<comment><USER9> Hey, Slate author here. I didn't realize this was going to get submitted to HN, Slate's still in beta but it's fairly production-ready.

You actually can handle things like "text-align", by adding custom data to nodes, which you can then read back via:

        const alignment = node.data.get('alignment')

Where .get(key) can be anything you want.

So text alignment would just be deciding on a key/value to use, and then toggling styles in your React component based on the alignment value.

That said, it doesn't handle text alignment "out of the box" if that's what you mean. It doesn't really handle anything out of the box, because you have to opt-in to anything that changes your schema. This is by design though, and it makes it a lot less complicated to use Slate than some of the other frameworks when you start building complex editors. But if you're looking for a drop-in editor that has basic features then Slate is not the right choice.

<comment><USER8> It's been several months since I used Draft, but if I remember correctly some of the editors based on Draft use same approach with defining custom data on the node and then applying custom class with assignment (for example you end up with: `<p class="align-right">Text Text Text</p>`). Too hacky for me. I would have loved if Nodes had native style suppord. Kind of like a Mark but applied to the whole root node instead of character ranges.

> it doesn't handle text alignment "out of the box"

That's quite alright. ProseMirror doesn't handle that out of the box too. A lot of new frameworks for rich editors surprisingly don't have that.

<comment><USER9> You can actually choose to use styles instead of classes fairly easily too. Since it's all defined in your custom React component you can do whatever you want, for example it would be written as....:

        function MyNodeComponent(props) {
          const { node } = props
          const textAlign = node.get('text_align')
          return
{props.children}

        }

Does that work for you? I'd be curious to hear what you mean by "marks but for nodes" and how you were thinking the API would look—sounds interesting!

Slate is different than Draft in that it gives you full control over the rendering logic—one of the things that frustrated me when using Draft and that I wanted to solve when creating Slate. It's much more flexible than Draft in that sense.

<comment><USER8> That does indeed work. You do need to override Node component and if you want to have even more changes - all of those would have to be located within that custom component.

I haven't really thought much about API or possible implementation for Marks for Nodes. My thoughts generally revolve around the idea that blocks should have extensible styling via Schema in the same way you can add custom Marks in your Schema. You should be able to change styling of the nodes within the Schema instead of overriding block simply because of the alignment or some other minor style.

<comment><USER9> That makes sense. And you can do that with Slate!

Since the components are all React components you can use the same types of extending patterns you'd use in React. In this case you'd probably use a higher-order component (HOC) that handles adding to the `style` property based on the node's data. And then you could wrap all of your other blocks with it...

        quote_block: alignment(QuoteBlock),
        code_block: alignment(CodeBlock),
        ...

One of the things I started with was making sure that components are just React components, so that all of the patterns that the React ecosystem brings can be used with Slate.

</comment></comment></comment></comment></comment></comment><comment><USER10> I just tried the example, and two things I noticed:

1\. The scroll region does not follow the cursor.

2\. I tried copy+pasting the demo-content a couple of times, and I noticed that the editor gets _really_ slow at relatively small document sizes. Especially "paste" is slow.

</comment><comment><USER11> I wish these libraries didn't invent their own document model when we already have HTML. Just let me edit HTML, within constraints setup by a schema, and allow extensibility of the document via HTML custom elements.

<comment><USER12> As other people mention, you can't really do this effectively via contenteditable, as user-intent (make that line bold) isn't uniformly transformed into consistent DOM across browsers. Medium had a great post about all that a couple years back. They wouldn't go through the trouble of re- implementing contenteditable if it weren't absolutely necessary.

There are other problems with using contenteditable as well. Copy paste becomes very challenging (you're sanitizing an dom tree and actively snipping stuff you don't want, instead of converting only things you do want into the intermediate representation). As an exercise, look at what an older tinyMCE editbox actually looks like on the inside after you paste something from MSWord.

I have been using quill.js in production for a while now (since well before it was 1.0), and having an internal representation of the text state that gets turned into HTML on the fly as necessary is a _good thing_. Since rich text editors are by necessity generating user content, I feel less worried about exposing crappy content on the DOM to people who view what was edited - I have total control over what DOM nodes get emitted (the legacy system I replaced allowed for pasting raw HTML, so I'm a little sensitive to this).

Plus, having my content be in an intermediate JSON representation is quite useful. For example: It's trivially easy to wire the source of diffs from quill into a log server for playback, so you can handle concurrent editing without an impossible amount of work. Also, I actually transform the representation differently depending on context. In an email notification, an atMention custom element gets rendered as a boring href with some style, but internally to the site, an atMention gets some custom element stuff that allows a flyover to show the user. This would be a lot harder if I didn't have control over the json -> html transformation at render time.

<comment><USER11> Who said anything about contenteditable or not having control over the content? The editor can have complete control and still store the document in markup rather than JSON.

HTML offers a number of advantages over a custom json-based document module, not the least of which is interop. You'd never want to store Skates data model in a database in case you switch editors.

<comment><USER12> I agree that HTML format has some strengths, but if you're working with data in javascript, there's a lot of strengths to a json format as well. Personally, I _do_ store json data in my database (in a jsonb postgres column, which gives me some query capabilities that aren't necessarily available operating on html strings in the database).

I also agree, though, that this leads to some degree of lock-in. You'd have to write a transformer to convert to your new framework, but that's not particularly hard.

Ultimately, though, the way I think about it is: react, angular, ember, vue all work by converting a JSON representation of state into a dom tree. So, an editor doesn't really need to be much different, as long as render times are fast, having an internal representation that's projected on to the DOM is a pretty common pattern.

</comment></comment></comment></comment><comment><USER13> Anyone wanting to look at Github, this is the link [https://github.com/ianstormtaylor/slate](https://github.com/ianstormtaylor/slate)

</comment><comment><USER14> Wow this looks really great. I've been building something with Draft.js + plugins and I can already see how this will make my vision easier to implement.

</comment><comment><USER15> Two probably dumb questions, but I can't tell from looking at the examples and a brief scan of the docs:

1) Can this be used as as content-editable substitute? 2) What format is the marked-up text stored in, and does it require Slate to be rendered?

<comment><USER9> Not dumb! (Author here.)

1) It depends on what you mean by substitute. It's kind of like a contenteditable that's been implemented with React components and an Immutable.js data model. It's not a drop-in replacement for contenteditable, since the APIs are different. But that is essentially what it lets you do.

2) The data is stored internally in an Immutable.js tree data model. You can convert it to JSON with the built-in `Raw` serializer. And you can define a schema with the `Html` serializer to convert it to HTML, which then you could render anywhere you want. I do this for one of my projects.

<comment><USER10> Does slate.js use contenteditable as an underlying primitive?

<comment><USER15> Yes. (That much I figured out). :-)

<comment><USER10> Thanks. Do you know why? If I had to write a text editor, I would not build it on top of something as unreliable (across browsers) as contenteditable. But I'm sure there is a good reason for it.

</comment></comment></comment></comment></comment><comment><USER16> Yay, inline markdown with applied styles. I'm missing that from most markdown editors. Sadly missing images and no real html output that I could apply my css on.

</comment><comment><USER17> I plan to use slate or draft to write MathJAX or ASCIIMaths writing. Can anyone suggest which one will be more better to write MathJAX or ASCIIMaths?

</comment><comment><USER18> How hard (or easy) is it to implement mentions?

<comment><USER4> Looks like this plugin will help [https://www.npmjs.com/package/slate- suggestions](https://www.npmjs.com/package/slate-suggestions)

</comment></comment><comment><USER19> The author mentions issues with a lot of editors he's not actually used, which I find a bit odd.

Still, I'm keen to give it a go.

<comment><USER9> Agreed! (Author here.)

I had originally started with just the few I'd used or researched in the initial stages. But folks opened issues in Slate asking for more comparisons, so I tacked on other libraries that people were interested in comparing. But I should really move those other libraries to another page in the docs somewhere with a better disclaimer.

</comment></comment><comment><USER4> How is the performance when rendering many read-only copies, e.g. for comments? I believe draftjs is quite poor.

</comment><comment><USER20> I get a 404 error for all of the examples.

<comment><USER9> Ah sorry about that. I'm the author of Slate, but I didn't submit this story or else I probably would have linked to the GitHub page itself. Fixing now!

<comment><AUTHOR> Ah, I submitted this, late last night (in my timezone), because I was happy to have stumbled across it on my quest for a highly-flexible rich text editor in React. Maybe I should have contacted you first. I linked to the docs because they're so clear and well made, and thought that might have been overlooked if I'd linked to slatejs.org. Hacker News search told me that the GitHub repo had been submitted twice in the past, with no results. Waking up I see that I even misspelled customizable... Sorry about that ^_^

<comment><USER9> No worries!! I should have fixed the intro docs page haha. I'm glad you like it and thanks for submitting!

</comment></comment></comment></comment><comment><USER21> Slightly off topic but does anyone happen to know if GitHub's markdown editor is open source?

<comment><USER22> I have been looking around for awhile for it. I found something that is inspired and looks pretty accurate.

[1] [http://code.weflex.org/markdown-editor- flavored/](http://code.weflex.org/markdown-editor-flavored/) [2] [https://www.npmjs.com/package/markdown-editor- flavored](https://www.npmjs.com/package/markdown-editor-flavored)

</comment></comment><comment><USER23> For people that don't know React. You got documentation in plain old js?

</comment><comment><USER24> All the example links broken, and show 404 not found

</comment><comment><USER25> I hate when pages break the back button on mobile</comment>


GCC optimized code gives strange floating point results - <AUTHOR> https://gcc.gnu.org/bugzilla/show_bug.cgi?id=323 <comment><USER2> I met William Kahan about 15 years ago and asked him about this. His response: "If your program says _double_ then yes, of course it's a bug if the compiler does long-double precision arithmetic instead. Isn't that obvious? Why are you even asking this?"

If the author of the IEEE 754 standard says that the way you do floating-point arithmetic is obviously a bug, you're probably doing it wrong.

<comment><USER3> And I believe he's right, if he was asked specifically what that program should do. If all the statements in the program are really just:

        #include <stdio.h>

        void test(double x, double y)
        {
          const double y2 = x + 1.0;
          if (y != y2) printf("error\n");
        }

        void main()
        {
          const double x = .012;
          const double y = x + 1.0;

          test(x, y);
        }

Then the comparison has to be done on doubles, and "but it's extended precision internally" mentioned in some answers is the wrong answer. All the variables in the program are stated to be doubles. If I'd write the x86 and x87 asm instructions I could easily write them so that they work as expected, even without SSE2.

However if you compile with some "special" optimizations that are documented to relax some specific code generation rules, then it can be OK that these specific optimizations behave differently, but then: you asked for it by choosing that documented optimization.

In the answers there, Richard Henderson wrote 2001-01-16 06:30:04 UTC:

"You are seeing the results of excess precision in the FPU. Either change the rounding precision in the FPCR, or work around the problem with -ffloat- store."

He's also right, if the -ffloat-store is documented to be turned off by -O and if it's documented that that flag also affects the precomputed calculations that are done in the compile time. If the second detail is not documented, the results could be unexpected, but again depends of what you expect from the optimizer. The properly written compiler and optimizer should perform the calculations during the code generation correct to the definition and not cut the corners. If the generated code was generated to actually perform the != during the run time, then the compile-time calculations are irrelevant, but the sane default should be something like -ffloat-store when not using SSE2 even with -O, if -O has the semantic "the optimizations that should not produce really unexpected effects." It sounds complicated, but it has its reasons.

P.S. answer to nsajko's post below: I mention -ffloat-store not ffast-math. Read the description of ffloat-store please:

[https://gcc.gnu.org/onlinedocs/gcc-3.2/gcc/Optimize- Options....](https://gcc.gnu.org/onlinedocs/gcc-3.2/gcc/Optimize-Options.html)

and also "Note on x86 and m68080 floating-point math" here [https://gcc.gnu.org/wiki/FloatingPointMath](https://gcc.gnu.org/wiki/FloatingPointMath)

<comment><USER4> With -O2 gcc actually compiles the program to

        xorl %eax, %eax
        ret

That is, it optimized away everything.

<comment><USER3> Yes, that's the correct optimization. That code obviously can't perform printf("error\n") to manifest the bug, and obviously both additions were made compile-time producing the results that are equal.

</comment></comment></comment></comment><comment><USER5> I remember that in university we had a test on a toy 16-bit floating point ALU. We had to walk through each of the steps in the ALU for the usual operations, showing the bit patterns at each step and keeping track of precision loss, etc. Because we had to track state and some aspects of the test were randomized, there were a few cases where identical operations on identical inputs resulted in different outputs due to the residual state.

During the next lecture, someone sticks their hand up and says, "this FPU thing is so nondeterministic, who in their right mind would use floating point for 99% of the kinds of work that we use computers for?"

The professor says, "you should think real hard about that the next time you decide to use a floating point type in your program."

<comment><USER6> > identical operations on identical inputs resulted > in different outputs due > to the residual state.

What? What residual state? Floating-point math is tricky, but on a particular machine/compiler/compile-options set you should expect that identical operations on identical inputs should give identical results.

What is this "residual state" you mention?

If you have an FPU that lets you change precision or rounding modes then you can get different results from the same operations - is that what you're talking about?

I get frustrated when people think that floating-point just randomly changes bits for no reason. It doesn't. It randomly changes bits for specific reasons, which can be understood.

<comment><USER7> trelliscoded probably meant bits in scratchpad, which are not accessible to the programmer. I read somewhere that those hidden bits may not be saved and regenerated properly as part of cpu resuming execution of the program (task switching). I do not know if that can happen, but even if not, achieving consistency on subsequent runs is quite difficult - apparently one has to make sure the program and all libraries it uses were compiled with special compiler flags that prevent data alignment influencing results of FPU operations.

[https://software.intel.com/articles/consistency-of- floating-...](https://software.intel.com/articles/consistency-of-floating- point-results-using-the-intel-compiler/)

</comment></comment></comment><comment><USER8> Probably a bug report with the most duplicates?

> The x87 FPU was originally designed in (or before) 1980. I think that's why > it is quite simple: it has only one unit for all FP data types. Of course, > the precision must be of the widest type, which is the 80-bit long double.

> Consider you have a program, where all the FP variables are of the type > double. You perform some FP operations and one of them is e.g. 1e-300/1e300, > which results in 1e-600. Despite this value cannot be held by a "double", it > is stored in an 80-bit FPU register as the result. Consider you use the > variable "x" to hold that result. If the program has been compiled with > optimization, the value need not be stored in RAM. So, say, it is still in > the register.

> Consider you need x to be nonzero, so you perform the test x != 0. Since > 1e-600 is not zero, the test yields true. While you perform some other > computations, the value is moved to RAM and converted to 0 because x is of > type "double". Now you want to use your certainly nonzero x... Hard luck :-(

> Note that if the result doesn't have its corresponding variable and you > perform the test directly on an expression, the problem can come to light > even without optimization.

> It could seem that performing all FP operations in extended precision can > bring benefits only. But it introduces a serious pitfall: moving a value may > change the value!!!

TIL long double is a thing.

<comment><USER9> So would turning off denormals fix the problem? That concept has given me 100 problems in the past and had helped me 0 times.

<comment><USER10> The solution is to compile with SSE2 on x86. (flags: -mfpmath=sse -msse -msse2)

On x86-64, the compiler should default to SSE2.

SSE2 is ~16 years old so compatibility shouldn't be an issue.

<comment><USER11> Technically, you only actually need the instructions from the original SSE set to do floating point operations. SSE2 adds a bunch of really useful integer floating point instructions.

But the only extra cpus that gets you is the Pentium III, AMD Athlon XP, and AMD Duron.

SSE2 is supported on every single x86 cpu released after those, such as the Pentium 4, Pentium M, and Athlon 64.

It's a real shame that people are still using CPUs that don't support SSE4, such as the AMD Phenom and Phenom II cpus, otherwise everyone would have moved to exclusive SSE4.

<comment><USER10> SSE1 is single-precision only. SSE2 added double precision.

So the bug will still appear for 'double' using just SSE1.

</comment></comment></comment></comment></comment><comment><USER12> In case anyone is wondering why this showed up right now, it's because tbrowbdidnso was reading the 'examples of unexpected mathematical images' article [1] and came across the one about FPU-dependent aliasing artifacts [2].

[1] [https://news.ycombinator.com/item?id=13736568](https://news.ycombinator.com/item?id=13736568)

[2] [http://www.cgl.uwaterloo.ca/csk/projects/alias/](http://www.cgl.uwaterloo.ca/csk/projects/alias/)

<comment><AUTHOR> Eh, was wondering how I wandered to the page myself.

</comment></comment><comment><USER13>
        *DIAGNOSTIC* THE TEST FOR EQUALITY BETWEEN NON-INTEGERS MAY NOT BE MEANINGFUL

UNIVAC 1107 FORTRAN IV error message for this, 1967.

</comment><comment><USER14> This is the bug that makes std::set< float > crash randomly under GCC with default compile options.

The fact that they consider this reasonable is insane.

<comment><USER15> Is having a set of floats reasonable?

<comment><USER14> Yes. There is nothing about floats that should make them unusable in a set.

In particular though, it's fairly common to use std::sets (or std::maps which use the same underlying data structure) to maintain something in a sorted order.

I ran into this myself when implementing A*. I was using an std::map with the float as key for the priority queue. Under GCC with default compile arguments it would randomly hang or crash due to GCC breaking the set invariant.

<comment><USER16> > Yes. There is nothing about floats that should make them unusable in a set.

Strictly speaking, there is. The < operator for floating-point types does not impose a _strict weak order_ because NaNs are a thing.

In any case, comparing floating-point values for strict equality (instead of using an appropriate epsilon) is more often a bad idea than not.

<comment><USER17> But std::set doesn't use operator< for comparing values, but _!(a <b) && !(b<a)_.

[http://www.cplusplus.com/reference/set/set](http://www.cplusplus.com/reference/set/set):

 _" Compare

A binary predicate that takes two arguments of the same type as the elements and returns a bool. The expression comp(a,b), where comp is an object of this type and a and b are key values, shall return true if a is considered to go before b in the strict weak ordering the function defines. The set object uses this expression to determine both the order the elements follow in the container and whether two element keys are equivalent (by comparing them reflexively: they are equivalent if !comp(a,b) && !comp(b,a)). No two elements in a set container can be equivalent. This can be a function pointer or a function object (see constructor for an example). This defaults to less<T>, which returns the same as applying the less-than operator (a<b)."_

I think that guarantees that std::set works for IEEE floating point values, and that NaNs are at the end of the set.

<comment><USER18> "I think that guarantees that std::set works for IEEE floating point values, and that NaNs are at the end of the set. "

No, this is, by definition, not a strict weak order because it's not transitive. It also doesn't have transitivity of equivalence.

It's at best partial ordering. "strict weak" order is, by the way, such an incredibly stupid name it's hard to know where to begin (it's what they call it in math, but it doesn't make it any more sensible)

It's much easier to understand in terms of the invariants.

Partial orders: f(x, x) must be false.

f(x, y) implies !f(y, x)

f(x, y) and f(y, z) imply f(x, z).

Strict weak ordering: All of the above _plus_ Define equivalence as f(x, y) and f(y, x) being both false. Then if x equivalent to y, and y equivalent to z, then x must be equivalent to z.

IE for all x, y, z: if f(x, y) and f(y, x) are _both_ false, and f(y, z) and f(z,y) are _both_ false, then f(x,z) and f(z, x) must _both_ be false.

It's strict because it's not reflexive, and it's weak because b < a && a < b does not imply a == b. It just implies they are "equivalent", and as mentioned, you get a total ordering on the _equivalence classes_.

</comment></comment></comment></comment></comment></comment><comment><USER19> Hah! That is such a great read, all of the "this isn't a bug" and the tombstones of all the "duplicate" bug reports. Reminds me how challenging it is to do decimal floating point in binary. Mike Cowlishaw could go on for literally _hours_ how just the concept is so broken and he wrote and shipped a decimal based floating point library which has none of these issues[1].

I still remember when I was told to never compare my floating point result to zero because even when it should be it probably won't be, always compare it to be less than a small sigma.

[1] [http://speleotrove.com/decimal/](http://speleotrove.com/decimal/)

<comment><USER20> > always compare it to be less than a small sigma.

Unfortunately, this approach can result in problems with the usual expectation that equality is transitive. Consider 0 < x < y < sigma. 0 == x, 0 == y, but x != y. This problem applies generally any time you're doing equality comparisons with sigma.

<comment><USER21> You aren't following the instructions. You have to rewrite _all_ comparisons to use a small sigma.

`0.f == x` becomes `SIGMA > x`

`0.f == y` becomes `SIGMA > y`

`x == y` becomes `SIGMA > abs(x - y)`

All the expression work if written correctly.

<comment><USER20> I probably should have given a better example instead of the hasty one. -sigma < x < 0 < y < sigma, abs(x - y) > sigma. I hope we can agree that this causes problems with transitivity :-)

There are _also_ problems with granularity. Suppose x and y are both much, much larger than sigma, and the smallest representable difference between x and y is e.g. 1.0. Now there's no possible way that abs(x - y) < sigma.

I suspect that the resolution to this is to compare equality within some number of ULPs, but: 1) You still have the transitivity problem and 2) This is not my field of expertise, and there may be a more appropriate solution.

<comment><USER21> > -sigma < x < 0 < y < sigma, abs(x - y) > sigma

This statement works fine. If the numbers are farther than sigma apart then `abs(x - y) > sigma` is true _as it should be_. It's not distance to 0, it's distance _from each other_ related to 0 that matters.

> There are also problems with granularity. Suppose x and y are both much, > much larger than sigma

This is part of the same problem. You have to choose a maximum acceptable value. And ensure you don't overflow it, and then you can choose a sigma based on that maximum value such that the difference between any two numbers never exceeds sigma. In practice you already have a maximum value for your domain that the simulation should never exceed or it means it is incorrect anyway.

</comment></comment></comment></comment></comment><comment><USER22> I have been following this bug in its various form for 30 years, and it is covered in pages 55-62 of The End of Error: Unum Computing. It is the "hidden scratchpad" bug where designers of a computing environment try to be helpful by providing more precision than what is explicitly requested by the programmer. If you read the blog postings of William Kahan about this issue, you will find he speaks fervently on both sides, sometimes defending it and other times vehemently deriding it. Thus the confusion. Kahan favors any technique that improves accuracy, even if it creates behavior that can only be explained through careful numerical analysis (which very few programmers have the time or skill set to perform).

Unums and posits, incidentally, are designed to eliminate all sources of irreproducibility and inexplicable behavior like this gcc bug. Hardware support for posits is on the way, maybe before the year is out. There is a way out of this mess, but it will take a few years to become mainstream.

<comment><USER23> No, Kahan advocates for what the C standard does which GCC doesn't implement correctly.

</comment></comment><comment><USER24> After reading much of the discussion, my conclusion is that it's past time to deprecate the x87 FPU. Ever since x86_64, it is expected that SSE2 will be available and compilers are expected to use it over the old FPU. Face it, this is an old Intel hardware "feature" that causes problems and has been obsolete for a long time.

</comment><comment><USER25> The -fexcess-precision=standard option, which is enabled by -std=c99 (requesting ICO C99 compliance), forces excess precision to be rounded away on all assignments and casts to narrower precision types.

See [https://gcc.gnu.org/onlinedocs/gcc/Optimize- Options.html#ind...](https://gcc.gnu.org/onlinedocs/gcc/Optimize- Options.html#index-fexcess-precision-960)

</comment><comment><USER26> [https://randomascii.wordpress.com/2013/07/16/floating- point-...](https://randomascii.wordpress.com/2013/07/16/floating-point- determinism/)

</comment><comment><USER27> Always a good read:

David Goldberg: What Every Computer Scientist Should Know About Floating-Point Arithmetic (1991)

[https://docs.oracle.com/cd/E19957-01/806-3568/ncg_goldberg.h...](https://docs.oracle.com/cd/E19957-01/806-3568/ncg_goldberg.html)

</comment><comment><USER28> I'm very surprised nobody has brought up John Gustafson's Unum work. His latest iteration is the posit, which he presented at Stanford earlier this month[0][1]

EDIT: apologies to CalChris[2]. What I should have said is _discussed_ Gustafson's Unum work.

[0] [https://www.youtube.com/watch?v=aP0Y1uAA-2Y](https://www.youtube.com/watch?v=aP0Y1uAA-2Y) (relevant question at 1h22m37s)

[1] [http://web.stanford.edu/class/ee380/Abstracts/170201.html](http://web.stanford.edu/class/ee380/Abstracts/170201.html)

[2] [https://news.ycombinator.com/item?id=13739617](https://news.ycombinator.com/item?id=13739617)

</comment><comment><USER29> Because I am doing a lot of numerical code in Fortran using the Intel compiler on Windows for production and GCC/GNU Fortran for prototyping on Linux I was a bit scared by the title. This is a problem with the equality comparison with doubles, something I try to avoid as much as possible, as this was the first thing I learnt to do 25 years ago... never compare double for equality.

Anyway, the bug report is linking to a very nice research paper[0]: "The pitfalls of verifying floating-point computations", which even so very technical is a good read on the subject.

[0]: [https://hal.archives-ouvertes.fr/hal-00128124](https://hal.archives- ouvertes.fr/hal-00128124)

</comment><comment><USER30> I think that this is a case of mismatch between two standards, not a bug. The C standard allows higher precision values to be used in place of lesser ones (extended precision vs double) and REQUIRES conversion to the correct precision during assignment (or casting). Also, the C abstract machine has the option to store those values when/if it deems opportune, it won't change observables as defined there.

On the other side, IEEE 754 allows extended precision to be used in place of double and of course requires that any chosen precision be kept or else.

But C Standard mandates IEEE 754 , too!

It seems to me that modern C deliberately chose to ignore such kind of mismatch in the name of (substantial) performance gains. K&R, good or bad, was way simpler!

<comment><USER31> > But C Standard mandates IEEE 754 , too!

Are you sure? I can't find it in the standard, specifically not in [1] page 28., section 22.214.171.124.2.

[1] (the final draft of C1X, dated 12 April 2011) [http://www.open- std.org/jtc1/sc22/wg14/www/docs/n1570.pdf](http://www.open- std.org/jtc1/sc22/wg14/www/docs/n1570.pdf)

<comment><USER30> You are absolutely correct, "mandates" is too strong a word. Annex F, which is normative, have a way out by not setting __STDC_IEC_559__, GCC6.3.0 does set a slightly different GCC_IEC_559. Yet I think that my argument still holds:

floating point calculations can be executed at wider range (§220.127.116.11.12);

assignments and casts are under obligation to wipe out that extra precision (§ same paragraph);

I am no language lawyer, but .. given the issue of program observable effects (§18.104.22.168.12) and the "as is" implied rule that governs optimizations, how possibly could equality be stable over time?

<comment><USER31> Yeah, you are right. It seems to be that the problem is that we cram two standards into too few types. Additionally to float, double, and long double there could be optional iec559_single_t, iec559_double_t and iec559_extended_t (similarly to stdint.h integer types). The operations on variables of these types could be strictly iec559 conformant.

</comment></comment></comment></comment><comment><USER32> For the PathScale compiler's initial release on x86, in the unlikely event that the x87 was used at all, I set -mpc64 as the default. We never got a customer complaint about it, despite having many numerically-savvy customers.

</comment><comment><USER33> More proof that floats are evil. Moral of the story is to never use equality to compare floats.

<comment><USER34> Floats are evil? o.O What do you propose instead if you want to deal with non- integer values?

<comment><USER35> There's always fixed point integers[0]. They're less flexible than floats, but at least the amount of precision available is much more clear and uniform, and all the rounding is pretty explicit.

[https://en.wikipedia.org/wiki/Fixed- point_arithmetic](https://en.wikipedia.org/wiki/Fixed-point_arithmetic)

[https://en.wikipedia.org/wiki/Q_%28number_format%29](https://en.wikipedia.org/wiki/Q_%28number_format%29)

<comment><USER36> Back in the late eighties, early nineties, I was of the opinion that fixed point is really the way to go, the only thing holding it back was lack of adequate compiler support. Until I realized what would actually happen if the world started using fixed point: programmers would always set the dynamic range too small, so we'd end up with a world full of programs that were constantly failing because they needed to be run on data larger than the test cases. The headaches of floating point, annoying though they can be at times, are much less onerous.

<comment><USER37> > Until I realized what would actually happen if the world started using fixed > point: programmers would always set the dynamic range too small, so we'd end > up with a world full of programs that were constantly failing because they > needed to be run on data larger than the test cases.

Floating point doesn't actually fix this issue. A fun bug I dealt with was a black screen caused by a few pixels having NaN values - which were then picked up by a blur pass which would poison basically the entire back buffer. The original NaNs were caused by divides by near-zero values causing out-of-range results.

You now add precision issues: Multiple coordinate systems are a common hack- around to maintain local precision when using single-precision based physics engines. Stuttering effects when time is represented as "floating point seconds since uptime" when integer wraparound would avoid any problem in the first place. And this is before even adding e.g. half precision or worse floats into the mix.

There's a reason floating point isn't used for a lot of serious banking calculations.

<comment><USER36> To clarify, any time I in any way praise floating point, I'm always talking about double precision. I think single precision floating point should never have been invented, and given that it unfortunately was, it's best to behave as though it doesn't exist. Don't get me started about half precision.

Financial calculations are another matter; they usually require decimal arithmetic, not because it doesn't give rounding errors, but because it gives _the same_ rounding errors an accountant gets when he checks the result with his desk calculator, and that's the requirement.

<comment><USER37> > To clarify, any time I in any way praise floating point, I'm always talking > about double precision.

64-bit fixed point solves a lot of problems too :)

> I think single precision floating point should never have been invented, and > given that it unfortunately was, it's best to behave as though it doesn't > exist. Don't get me started about half precision.

A 4x memory bandwidth & storage penalty for HDR buffers would be absolute murder (bloating a single 4k RGBA surface to 265MB if I've done my math right), so I'm glad we have the option. And in the historical context lower precision options were invented in, RAM wasn't nearly as cheap as it is today. Tradeoffs.

</comment></comment></comment></comment></comment></comment></comment><comment><USER38> I have found that I often end up forsaking the performance and using a decimal type. It is definitely slower, but the correctness is often more desirable.

<comment><USER26> A friend mentioned being in a meeting where Mike Cowlishaw argued that Javascript's floating point type should be decimal floats not binary floats. Everyone including my friend thought he was out to lunch. My friend says now, 'a great opportunity was pissed away'

Because almost no one actually wants wants binary floating point.

<comment><USER6> Why not? Decimal floats have definite advantages if you are dealing with decimal data, but that is not the only type of data that floats are used for, and it probably isn't the majority. Binary floating point is better than decimal floating point if you don't care about decimal-interop (more precision per bit, more consistent precision), and decimal floating point has all the same issues (except for confusing conversions to decimals).

So, if you are dealing with raw sensor data, neural nets, character models in a game, numerical simulations, and many other types of data, the extra precision of binary floating-point is far more important than any confusion that arises during those (rare) conversions to decimal.

<comment><USER38> It's all about costs and benefits though. I do think that a decimal type should be standard along side a float so that the most appropriate one can be chosen. But in many contexts, one needs to go outside the immediate ecosystem to third party libraries to get an alternative to floating point. Currency is a good example of a type that should not use floating point arithmetic but is far too often.

<comment><USER26> I think the difference is binary floating point tends to be on massive automatically generated sets of data. And binary floats are mostly perfect for that.

However the bias towards scientific computing is a problem. because there are large numbers of use cases where the inputs and outputs are decimal floats because you're dealing with humans and their preference for base 10.

</comment></comment></comment></comment></comment><comment><USER39> This is well-known by everyone who works with float determinism. You have to set the FPU flag with assembler to force the desired width (among other things). See: fldcw, LDMXCSR, etc.

Take into account rounding modes and denormals too, and make sure to disable float optimisations that break determinism in your compiler.

</comment><comment><USER40> > Regardless of where one wishes to put the blame, this problem will _not_ be > fixed. Period.

I realise that floating-point numbers are, to use the technical term, "whack", and I also realise that triaging the issue seems to point to a hardware error, not a software error, but I can't get passed this "yeah, there is a problem, so what?" attitude. Am I missing something critical here that makes this unreasonable to work around or fix?

<comment><USER41> According [indirectly] to one of the comments, the issue is that there is no solution which solves all the problems in the context.

Reference:

[https://gcc.gnu.org/ml/gcc/2003-08/msg01257.html](https://gcc.gnu.org/ml/gcc/2003-08/msg01257.html)

Specifically:

        There are two things missing. The abililty to turn on the workaround in c (i.e. emit FP rounding instruction after every operation), and a bug fix for the register spills.

        Even with those things, I think we are still in trouble. In the first case, having explicit rounding instructions eliminates the excess precision problem, but it introduces a double-rounding problem. So we have lost again there. This is probably not fixable without changing the hardware. In the second case, fixing the problem with reload spills eliminates one source of unpredictable rounding, however, we still have the problem that local variables get rounded if they are allocated to the stack and not rounded if they are allocated to an FP reg-stack register. Thus we still have the problem that we get different results at different optimization levels. So we still lose there again also. This might be fixable by promoting all stack locals to long double to avoid unexpected rounding, but that will probably cause other problems in turn. It will break all programs that expect assignments doubles will round to double for instance. If we don't promote stack locals, then we need explicit rounding for them, and then we have double-rounding again.

        I really see no way to fix this problem other than by fixing the hardware. The hardware has to have explicit float and double operations.

</comment></comment><comment><USER42> Comment 109 has a good exposition I'll just copy here:

Jan Lachnitt 2008-05-20 16:59:31 UTC

I also encountered such problems and was going to report it as a bug in GCC... But in the GCC bug (not) reporting guide, there is fortunately a link to this page and here (comment #96) is a link to David Monniaux's paper about floating-point computations. This explains it closely but it is maybe too long. I have almost read it and hope I have understood it properly. So I'll give a brief explanation (for those who don't know it yet) of the reasons of such a strange behaviour. Then I'll assess where the bug actually is (in GCC or CPU). Then I'll write the solution (!) and finally a few recommendations to the GCC team.

EXPLANATION

The x87 FPU was originally designed in (or before) 1980. I think that's why it is quite simple: it has only one unit for all FP data types. Of course, the precision must be of the widest type, which is the 80-bit long double. Consider you have a program, where all the FP variables are of the type double. You perform some FP operations and one of them is e.g. 1e-300/1e300, which results in 1e-600. Despite this value cannot be held by a "double", it is stored in an 80-bit FPU register as the result. Consider you use the variable "x" to hold that result. If the program has been compiled with optimization, the value need not be stored in RAM. So, say, it is still in the register. Consider you need x to be nonzero, so you perform the test x != 0. Since 1e-600 is not zero, the test yields true. While you perform some other computations, the value is moved to RAM and converted to 0 because x is of type "double". Now you want to use your certainly nonzero x... Hard luck :-( Note that if the result doesn't have its corresponding variable and you perform the test directly on an expression, the problem can come to light even without optimization. _It could seem that performing all FP operations in extended precision can bring benefits only. But it introduces a serious pitfall: moving a value may change the value!!!_

WHERE'S THE BUG

This is really not a GCC bug. _The bug is actually in the x87 FPU because it doesn 't obey the IEEE standard._

SOLUTION

The x87 FPU is still present in contemporary processors (including AMD) due to compatibility. I think most of PC software still uses it. But _new processors have also another FPU, called SSE, and this do obey the IEEE_. GCC in 32-bit mode compiles for x87 by default but it is able to compile for the SSE, too. So the solution is to add these options to the compilation command: -march=* -msse -mfpmath=sse

Yes, this definitely resolves the problem - but not for all processors. The * can be one of the following: pentium3, pentium3m, pentium-m, pentium4, pentium4m, prescott, nocona, athlon-4, athlon-xp, athlon-mp, k8, opteron, athlon64, athlon-fx and c3-2 (I'm unsure about athlon and athlon-tbird). Beside -msse, you can also add some of -mmmx, -msse2, -msse3 and -m3dnow, if the CPU supports them (see GCC doc or CPU doc).

If you wish to compile for processors which don't have SSE, you have a few possibilities:

(1) A very simple solution: Use long double everywhere. (But be careful when transfering binary data in long double format between computers because this format is not standardized and so the concrete bit representations vary between different CPU architectures.)

(2) A partial but simple solution: Do comparisons on volatile variables only.

(3) A similar solution: Try to implement a "discard_extended_precision" function suggested by Egon in comment #88.

(4) A complex solution: Before doing any mathematical operation or comparison, put the operands into variables and put also the result to a variable (i.e. don't use complex expressions). For example, instead of { c = 2 _(a+b); } , write { double s = a+b; c = 2_ s; } . I'm unsure about arrays but I think they should be OK. When you have modified your code in this manner, then compile it either without optimization or, when using optimization, use -ffloat-store. In order to avoid double rounding (i.e. rounding twice), it is also good to decrease the FPU precision by changing its control word in the beginning of your program (see comment #60). Then you should also apply -frounding-math. (5) A radical solution: Find a job/hobby where computers are not used at all.

<comment><USER42> On a related note, testing floating point equality should be known as a recipe for unexpected results.

</comment></comment><comment><USER43> [https://docs.oracle.com/cd/E19957-01/806-3568/ncg_goldberg.h...](https://docs.oracle.com/cd/E19957-01/806-3568/ncg_goldberg.html)

</comment><comment><USER44> This idea is related to the strictfp modifier in Java. [https://en.wikipedia.org/wiki/Strictfp](https://en.wikipedia.org/wiki/Strictfp)

</comment><comment><USER45> As a system administrator, my favorite part of the Bug 323 thread is the mail bounces from postmaster.

</comment><comment><USER46> Should add the date to the title (2000)

</comment><comment><USER47> People need understand that floating point comparison at floating precision is _undefined_ , so it is not a bug. The proper floating number representation should track its precision. Since that is more complicated and potentially performance affecting, so until then the programmer should manage the precision tracking on their own (similar to memory allocations), that is, one should do:

      if ( f_a < f_b-f_eps)
      if ( f_a > f_b+f_eps)
      if (fabs(f_a-f_b)<f_eps)

If you don't do this, then you are on your own. Sometimes it is fine, but sometimes it may give you grief, just like the dangers of other undefined behaviors.

PS: I also have a problem that people think "undefined behavior" is _undefined_ because certain standards say so. Undefined behavior is due to lack of agreement of common sense or defies common sense. For example, divide by zero, there is no common sense doing so, and define its behavior does not change anything. Signed integer overflow, there is no common sense agreement for that behavior either. The common sense for undefined behaviors is to avoid it. (On the other hand, there seems to be some common sense agreement on unsigned integer overflow, and people are writing code based on that behavior. But that agreement is weak, so I would avoid it if I can regardless what the standards say.)

PS2: The reason that floating point comparison is undefined behavior because, unlike fixed precision or integers, floating point number has floating precision. Floating precision does not have common sense correspondence. In any real application (where common sense is based), a finite, and above all, consistent precision, is always assumed.

And the solution, by manually specifying a f_eps, we put that comparison into a common-sense understanding that you know the result of comparisons is only meaningful above f_eps. With that common sense, having an internal calculation in higher precision is always OK -- isn't that common sense?

PS3: The question is why not turn undefined behaviors into errors? Undefined behavior does not mean there are no behaviors, and the behaviors can be used, or more often, ignored to achieve certain efficiency. When programmers write code with undefined behaviors, they are extracting their code out of common sense domain, but into a special sense domain. Special sense domain is practically fine. Everyone has their own efficient quirks, just do not expect your quirks to be understood or supported by out large where common sense rules (unless you carefully define it and document it). A language that allows for undefined behaviors allows for individual efficiency, which sometimes it is rather good.

For example, it is ok to write programs with direct floating point number comparisons as long as you understand that your program does not care when the comparison falls into the ambiguous range. Only you who understand your special application can assess that. If you are the only one who is responsible for the result of that code, then it is perfectly fine!

<comment><USER48> Essentially everything you say here is false.

Floating point numbers represent rational numbers, and their arithmetic is straightfoward and follows common sense: you compute the exact operation using rational numbers, and the result is the closest floating point number available.

Floating point operations are never "undefined" either, they are perfectly deterministic and completely specified by the IEEE 754 standards.

There are three floating point values that do not correspond to rationals: INF, -INF and NAN. There is also "-0" that corresponds to the same rational as "0". The arithmetic of these values is extremely common sense to anyone that knows some calculus: 1/0 = INF, INF*0=NAN, INF+INF=INF, INF-INF=NAN, etc.

There is no ambiguity whatsoever in floating point numbers.

<comment><USER22> You say 1/0 = INF, but the 754 standard says negative zero is different, and 1/(-0) = -INF. Yet it says -0 is numerically equal to 0. It also specifies that the square root of -0 is -0. This is neither straightforward nor common sense.

They also do not round to the nearest representable rational number. If you double MAXREAL, it will round to INF, even though MAXREAL is infinitely closer to 2*MAXREAL than INF is.

Despite efforts in the 2008 update to the 754 Standard, there still is no guarantee that the Standard will produce reproducible, bitwise identical results. The Standards committee admits this. It is important for everyone to understand the shortcomings of the Standard.

<comment><USER48> Well, "common sense" is not the same for everybody then. For a mathematician, the rules 1/0 = INF, 1/(-0) = -INF are perfectly straightforward and common sense (if you think about the limits).

Regarding sqrt(-0)=-0, I concede it looks a bit strange. Yet, as explained in the famous Kahan article "much ado about nothing's sign bit", it turns out to be natural when you work with complex numbers.

As for the nondeterminacy, I suppose you are talking about transcendental functions? The basic arithmetic should be bitwise reproductible.</comment></comment></comment></comment>


How CloudFlare extracts a signal from 10 trillion log lines each month [video] - <AUTHOR> http://www.thedotpost.com/2015/06/john-graham-cumming-i-got-10-trillion-problems-but-logging-aint-one <comment><USER2> Good god, 4 million log requests per second. 400 TB a day (compressed) if they stored the logs.

I recently setup a fun project using a RethinkDB cluster and Node.js express middleware[1] that logs all requests to RethinkDB in JSON. I did some load testing and sustained 1,400 writes per second, and was quite happy thinking this would scale up very larger. However, not 4 million write per second large. :-)

[1] [https://github.com/commando/express-rethinkdb- logger](https://github.com/commando/express-rethinkdb-logger)

<comment><USER3> It would be a better comparison if we knew across how many nodes they're sustaining that. Maybe they just have 4,000 nodes doing 1k TPS each.

<comment><USER4> I don't really think the word "just" belongs in that sentence ;)

</comment></comment></comment><comment><USER5> Interesting how NGINX + Lua is becoming more and more widely used in mission critical applications with huge amount of traffic. Since the introduction of LuaJIT performances have been outstanding and many companies like Netflix, Alibaba, Cloudflare, Kong [0], Airbnb all run on a customized nginx with Lua modules; doing amazing things from security to API management.

[0] [http://github.com/mashape/kong](http://github.com/mashape/kong)

<comment><USER6> I predicted about a year ago that nginx/LuaJIT (OpenResty) is the sleeping giant of web development. I've seen more and more companies start using it, and I wouldn't be surprised if people start talking about it as a Node.js alternative in the not so distant future.

<comment><USER7> HAProxy will be using Lua in future releases for similar reasons. Interesting trend!

<comment><USER5> And Lua is taking off because of these reasons such as the ability to extend Nginx or HAproxy. Plus, is simple to use, easy to learn and highly efficient, without the need to touch C/C++.

</comment></comment></comment></comment><comment><USER8> What is the deal with the recent "thedotpost.com" envelope for perfectly valid YouTube URLs? I want to flag this post just for that.

[https://www.youtube.com/watch?v=LA- gNoxSLCE](https://www.youtube.com/watch?v=LA-gNoxSLCE)

<comment><USER9> jgrahamc is the speaker, it was at a conference run by the same company as thedotpost.com.

I'm not really seeing the issue. Complaining about how someone chooses to link to their own content seems silly tbh.

<comment><USER8> My apologies, I just noticed a flood of these previously unrecognized domains which had a lot of "web chrome" around a central YouTube video. I shouldn't have included the language about flagging it, as that made my comment more negative than I intended it. I am slowly learning to pare down my HN comments to avoid that editorializing.

<comment><USER10> In this particular case, the website logo is the same logo on the speaker's podium and on the wall behind the speaker :)

</comment></comment></comment></comment><comment><USER5> That is 4M requests per second. To put this in prospective, the established Akamai runs at 25M/s ([http://www.akamai.com/html/technology/real-time-web- metrics....](http://www.akamai.com/html/technology/real-time-web- metrics.html)), which means that Cloudflare is quickly growing to hold 20% of Akamai traffic.

</comment><comment><USER11> [https://github.com/cloudflare/jgc- talks/blob/master/dotScale...](https://github.com/cloudflare/jgc- talks/blob/master/dotScale/2015/10trillion.pdf)

I like that github extracts the slides for me, but what would be better is if it could extract just the plain text.

</comment><comment><USER12> The Nginx + LuaJIT approach is new to me. I went looking and found their blog post about it [1]. That post mentions log aggregation, but it sounds from this talk like they're doing dynamic routing via the Lua code. Is the idea to accept requests for entirely separate sites via one front-end Nginx host(s), and then perform a sort of NAT or virtual routing to different client properties based on the request headers?

At any rate, very interesting stuff, and more to read up on now.

[1]: [https://blog.cloudflare.com/pushing-nginx-to-its-limit- with-...](https://blog.cloudflare.com/pushing-nginx-to-its-limit-with-lua/)

</comment><comment><USER13> In the world of lossy counting. This [1] could be something to look at if you wanted to answer the question "how many requests/secs towards jgc.org at noontime, two months ago"

[1] [https://github.com/dgryski/hokusai](https://github.com/dgryski/hokusai)

<comment><USER6> Why not HyperLogLog?

<comment><USER14> Presumably because there's no point replying to a video talk about HyperLogLog with a comment saying "you might want to look into HyperLogLog".

</comment></comment></comment><comment><USER15> No love for Elasticsearch :/

<comment><USER8> Are you complaining about the obvious ES JSON in one of the slides not getting a mention or just the lack of mention at all?

If the latter, it's highly likely due to the findings (possibly even independently verified) of the treatment ES received from Jepsen, which was revisited in a talk at that same conference: [https://news.ycombinator.com/item?id=9778291](https://news.ycombinator.com/item?id=9778291)</comment></comment>


Ex-Googler: 'Tons of Engineers' Want to Leave Google - <AUTHOR> http://www.businessinsider.com/ex-googler-tons-of-engineers-want-to-leave-google-2015-1 <comment><USER2> "During a trip to Palo Alto earlier this week, we spoke with another high- profile venture capitalist who said he also expected talent to flee Google in increased numbers during 2015 — but only if the company is unable to get its stock growing again"

So the alternative to a sorta comfy job with lots of free perks is to slog insane hours in a startup with absolutely no guarantee of a payout? I'd love to know who this VC is; in any case, this is shoddy journalism. It makes sweeping generalization of all employees being after money, which some might be true for some, but I'm sure many of the ones leaving love to challenge themselves to work in unexplored areas, something which no big company provides (exception: GoogleX and other skunk works outfits which let's face it, you gotta know the right people to get in).

</comment><comment><USER3> I've heard this story before, but I don't remember a mass exodus from "da goog" as the article hideously calls it. I wonder what the motive is.

<comment><USER4> Reads like FUD, also it is from January 2015.

<comment><USER5> I'm pretty sure this exact story made it to HN when it was published as well - I remember reading this then.

</comment></comment></comment><comment><USER6> It's funny to click on the linked Tweet and see the person they're quoting say:

"Or the part where my dumb tweet became a scandal on BI?"

</comment><comment><USER7> Meh. Google has what, 20,000 engineers? So even a normal turnover rate is thousands every year. This is a "dog bites man" story.

<comment><USER8> It's about the trend and the environment not the numbers.

<comment><USER6> They're 5 years late. 2009/2010, the trend was ex-Googlers leaving for startups. When I left in 2014, the trend was ex-Googlers leaving for public- sector service or academia, which we'll probably hear about in another year or two, and who knows where it is now?</comment></comment></comment>


The FBI's Scientology investigation - <AUTHOR> http://www.tampabay.com/news/scientology/article1270036.ece

<comment><USER2> For 5 years I lived in a house in Los Angeles, where one of the previous residents had been a member of Sea Org - almost 10 years before we moved in. We would receive a piece of mail addressed to him from Scientology almost every single day we lived there. Posters, letters, DVDs you name it. We'd collect the mail and pile it on the floor as entertainment for friends. They probably sent close to $200 worth of mailers to that house every year. People from Scientology would come to the house asking for this guy, and we'd tell them he hadn't lived there in years. They'd return, leave handwritten notes on the door. In the storage area under the house were boxes of his family's clothes and belongings, I wouldn't be surprised if they had just abandoned their things and fled. Scientology is no joke.

</comment><comment><USER3> I was born and raised here in Pinellas County with Scientology World headquarters. I still live here in Tampa, and I've known a variety of Scientologists, and done business with the Church.

They've been this way since before I was born; this is nothing new. I have yet to see a news article accurately report on them, their beliefs, their activities, or their crimes, which are legion.

The St. Pete. Times used to war with them regularly for years; not once did I read an article that didn't have numerous sloppy mistakes that basic research would have caught.

They really are a religion: they think they're saving souls and the World, and anything they do is worth it; the ends justify the means. Since the World hangs in the Balance, and they're the warriors on the front lines, they're above the law.

What's worse is a lot of their minor techniques are quite effective, especially on drug addiction and the like, which convinces many people that they're the One True Path.

</comment><comment><USER4> I find it interesting that "freedom of religion" helped them out of a case. Why should doing something that'd otherwise be illegal suddenly change because you claim it's "religious"?

Freedom of religion seems like a relic of an older time, or lack of clear thinking. If you have freedom to think, assemble, speak, why does religion need special casing? How is it different than having "freedom for hobby clubs"?

<comment><USER3> Because historically, nations have been eager to take advantage of Religion's power to brainwash. They have found every way they could to worm religion into their toolbox. "Congress shall make no law respecting an establishment of religion, or prohibiting the free exercise thereof" prevents the government from saying we all have freedom to assemble, speak, etc, but the gov't is going to preferentially support this particular one over here. Or a myriad of such tricks history has blessed us with.

</comment></comment><comment><USER5> I enjoy the content of this link, but I don't know what this has to do with hacker news. This is a great link for reddit, or other like site. Hacker News has been losing its focus on technology articles.

<comment><USER6> "On-Topic: Anything that good hackers would find interesting. That includes more than hacking and startups. If you had to reduce it to a sentence, the answer might be: anything that gratifies one's intellectual curiosity."

<http://ycombinator.com/newsguidelines.html>

<comment><USER5> So basically absolutely anything that is not internet memes is on the table. I would love it if Hacker News had a section of their site that was targeted specifically to news about programming. Then I could treat reddit as reddit and hacker news could have the programming news I want.

I have been following hacker news for years, and it seems that the subject of the articles has been slowly straying from the topic of programming to more general interest. I don't mean to pick on this article specifically.

<comment><USER7> In a way, I think making a programming only section was what turned reddit into the non-programming centric site it is now.

If you maintain one view, there is enough pressure on the main page to sustain the programming articles rather than just say "go to the programming section".

</comment></comment></comment></comment><comment><USER8> I wonder if some of their techniques could be used for a hacker boot camp. Voluntarily subject yourself to psychological abuse in order to get rid of distractions and achieve "the zone" or high productive states. Great for those experiencing longer than usual spells of burn-out. Mind hacking you back into shape.

<comment><USER9> I don't think that abuse would be an effective way to get someone into the zone. In fact, I think it would be a good way to keep people _out_ of the zone.</comment></comment>


Cryptocurrencies have crashed 20% in two days - <AUTHOR> https://techcrunch.com/2017/09/04/cryptocurrencies-have-crashed-20-in-two-days/ <comment><USER2> It's absurd to call it a crash at this point. It's just barely a bear market.</comment>