Mathematicians bring ocean to life for Disney's 'Moana' - <AUTHOR> https://phys.org/news/2017-01-mathematicians-ocean-life-disney-moana.html <comment><USER2> We watched Monana a couple of weeks ago and afterwards my wife said that she was amazed with how good the water looked. I said, I bet there were a bunch of mathematicians working on that :-)

In another life I'd be doing maths for 3D movies. I went to university in Wellington where Weta are based (Peter Jackson's film companies) and at one point had a look around the digital arm. I seem to recall that they had the SGI machines as desktops and a big unix rendering farm. I just love the way you can use maths to create such magic.

Anyway, if you haven't seen it yet, Moana is fantastic. Incredible animation, catchy pop hooks in the music and a brilliant storyline. It's wonderful to see a Disney film with a strong female lead that needs neither male support nor a love interest.

<comment><USER3> > a strong female lead that needs neither male support nor a love interest

Now I'm trying to think of other Disney leads of _either_ sex without a love interest... is there one? Other than those for whom it's not really applicable (ones who are too young, mostly).

<comment><USER4> Judy (Zootopia) Hiro (Big Hero 6) Elsa (Frozen) Ralph and Velellope (Wreck it Ralph)

There are plenty more. I think it has more been current society that is recognizing it more.

<comment><USER5> _Wreck it Ralph_ puts the love interest relationship between Felix and Calhoun (and subverts it all over the place).

Those are all quite recent, while the historical Disney films all fall into the traditional template of male and female lead falling for each other. To the extent that _Enchanted_ can assume the audience knows the formula that they're parodying.

There is indeed a recognition and a policy change that stories don't have to be made to fit that formula any more. And it seems that the films are exploring other affectionate relationships, such as the sisters in _Frozen_. _Wreck it Ralph_ clearly has _affection_ between Ralph and Vanellope, but it's more a parental/big-brother-little-sister thing than a romance. _Brave_ focuses on mother/daughter. And so on.

</comment></comment></comment></comment><comment><USER6> "Solve, or nearly solve, partial differential equations". That's game physics. Most of the effort is to come up with ways to "nearly solve" without having awful stuff happen. There's a long history of awful stuff, going back to when Seamus Blackley botched Trespasser in 1998.

Game physics still tends to go "boink", because with impulse/constraint collisions, everything, including large, heavy objects, bounces instantaneously. Stuff flying apart, though, is rare now; most systems drain the energy out of a system when they detect that happening. It's physically wrong, but looks less awful.

I used to work on this stuff. I solved the "boink" problem for articulated rigid body physics in the 1990s, but couldn't make it work in real time on 100MHz CPUs.

<comment><USER7> > Stuff flying apart, though, is rare now; most systems drain the energy out > of a system when they detect that happening. It's physically wrong, but > looks less awful.

Most games are using position based dynamics now (see Nvidia PhysX) so there isn't even a variational principle to arrive at the system from an energy. Just a constraint satisfaction problem with a weak notion of momentum coupled in.

</comment></comment><comment><USER8> Also coming into play during Moana is the hyperion rendering engine, which was first showcased in Big Hero 6:

[https://www.disneyanimation.com/technology/innovations/hyper...](https://www.disneyanimation.com/technology/innovations/hyperion)

Also, here's an article containing some stills from the movie:

[http://blog.yiningkarlli.com/2016/11/moana.html](http://blog.yiningkarlli.com/2016/11/moana.html)

<comment><USER9> Thank for sharing - never been on DA website.

The whole blog of publications is super interesting!

[https://www.disneyanimation.com/technology/publications](https://www.disneyanimation.com/technology/publications)

<comment><USER4> Glad you like it. I work at Disney Animation. I have been pushing to get more "Innovations" posted but it's not always the easiest thing to get "secret sauce" published. [https://www.disneyanimation.com/technology/innovations](https://www.disneyanimation.com/technology/innovations)

I also helped push this forward to get other divisions to think about open sourcing code more often. [http://disney.github.io/](http://disney.github.io/)

</comment></comment></comment><comment><USER10> I was looking at the animation referenced in the article of water flowing over a rocky terrain and couldn't help but notice almost life-like and pixel- perfect precision of collision detection between water particles and the rocky surface. Yet, in modern gaming, while it's improved a great deal, quite often you'll see surfaces "fusing" together before a collision is detected. I think there is a more technical term for this, but it's not coming to mind. Is this just a matter of usability vs performance?

<comment><USER11> The movie is using a whole compute cluster to do a physics simulation when rendering the movie. Your computer has 1 CPU and 1 GPU.

Your computer is just very limited in how much attention to detail it can have, because it has wildly less processing power _and_ has to get its job done in real time.

By contrast, Disney is using a server farm to spend 3 seconds analyzing the mechanics of a tuft of hair wiggling in the breeze for 1 second.

<comment><USER12> Has anyone tried to make a real game in the same way?

Ie a game which was hosted on a remote desktop that had direct access to a GPU server farm on a very fast network.

<comment><USER2> Not a field I work in but I guess it would never work. You'll have the network latency to contend with. Or people would need to self host a server farm, which probably makes your market pretty small :-)

<comment><USER13> I wouldn't say never. The golden rule of computer games code is that you get to cheat _a lot_.

</comment></comment></comment></comment></comment><comment><USER14> I've been waiting decades to play a surf sim that utilizes proper fluid dynamics. Maybe before I die it'll happen.

<comment><USER15> Wave Race 64 had plausible wave physics. Could probably go much further with the hardware of today.

</comment></comment><comment><USER16> Why is it that we can accept expressive/cartoonish looking characters but need to have the most realistic environmental rendering (light, physics effects)?

I challenge the studios to be imaginative and equally expressive on the latter. Give us the miyazaki equivalent in 3d animated films.

Don't take fthe easy way out by emulating reality. Put true art into it

<comment><USER17> My guess is the uncanny valley[0] is still too big for "realistic" human characters. As for the goofy-looking chicken, it's convenient to make an animal goofy-looking for comic relief. (And in the case of Jiminy Cricket, I think Disney would rather draw, and people would rather look at, a humanoid with a weird head than an actual insect.)

[0] "the uncanny valley is the hypothesis that human replicas that appear almost, but not exactly, like real human beings elicit feelings of eeriness and revulsion among some observers" [https://en.wikipedia.org/wiki/Uncanny_valley](https://en.wikipedia.org/wiki/Uncanny_valley)

</comment></comment><comment><USER18> for the EU viewers that are confused: in some countries the title and the character's name is Vaiana.

<comment><USER19> moana pozzi was a famous italian porn actress in the '80, that made also standard tv programs and involved in politics, too. I'm pretty sure she was famous in Europe too.

</comment></comment><comment><USER20> I wish the article went more into detail about what is novel about the algorithms they worked on.

<comment><USER21> [https://www.math.ucla.edu/~jteran/papers/JSSTS15.pdf](https://www.math.ucla.edu/~jteran/papers/JSSTS15.pdf)

</comment></comment><comment><USER7> The hair was also great in Moana. It is fun to see nonlinear Cosserst rod theory in use on the big screen!

</comment><comment><USER22> The Taichi library linked [1] on HN recently has a MIT licensed APIC implementation here :

[https://github.com/IteratorAdvance/taichi/blob/master/includ...](https://github.com/IteratorAdvance/taichi/blob/master/include/taichi/dynamics/apic.cpp)

[1]: [https://news.ycombinator.com/item?id=13325190](https://news.ycombinator.com/item?id=13325190)

</comment><comment><USER23> This is really cool. I wonder how one goes about getting into work like this - niche field but few qualified to work it, I'd guess.

<comment><USER24> You can come do a PhD with me at Waterloo. #shamelessplug [https://cs.uwaterloo.ca/~c2batty/](https://cs.uwaterloo.ca/~c2batty/)

<comment><USER23> Small world! I actually am majoring in comp sci and physics. I'll definitely message you if I end up getting into research next year.

</comment></comment></comment><comment><USER25> Scarily realistic: [https://www.youtube.com/watch?v=3mULoRzfFcE](https://www.youtube.com/watch?v=3mULoRzfFcE)

</comment><comment><USER26> A little off-topic, but this site's cert appears to have a revoked intermediate authority. Pic: [http://i.imgur.com/lIseT66.png](http://i.imgur.com/lIseT66.png)

But no-one else seems to be complaining about this so I wonder if it's just me?

<comment><USER27> All fine for me in Safari on OSX 10.10.3 [http://i.imgur.com/ZwbQ08N.png](http://i.imgur.com/ZwbQ08N.png)

</comment></comment><comment><USER28> In the video, it is mentioned several times "stability" and "noise". Can someone explain what do these two terms refer to in the sense of particle simulation?

</comment><comment><USER29> "where they used science to animate snow scenes"

Cringes

<comment><USER30> Well, they did, in a very non-buzzwordy manner.

</comment></comment><comment><USER31> This is nice from an artistic perspective. But from a computational physics perspective, it feels like cheating. How much of this is realistic? And are we now encouraging more and more people to focus on "fake" physics because it makes it somehow easier to produce something cool? I sure hope not.

<comment><USER32> It's not all about looking cool, it's about coming up with algorithms that are computable in a reasonable amount of time while also behaving as similarly as it is possible to interactions of objects in the real world.

Faking physics is a mastery. Trade-offs are unavoidable.</comment></comment>


C++ State Machines - <AUTHOR> http://beza1e1.tuxen.de/cpp_state_machines.html <comment><USER2> Simulink, and specifically stateflow, is commonly used in the aerospace industry to graphically create state machines that can be autocoded into high performance C or C++ code. I use simulink and stateflow everyday to create flight control software. I feel like more software engineers should learn how to use graphical coding tools, as IMO it's so much easier to represent complex state machines graphically.

It's a shame that Mathworks is essentially the only game in town though, and their software licenses are fairly expensive.

<comment><USER3> I'd rather have tools that can take code which implements a state machine and produce a statechart than go the other way around. There are a few tools which will go from statecharts to code and they've been around a while but never seemed to get very popular. I think dealing with a complex system as a statechart is potentially harder than dealing with code.

That said, my friend works on satellite software implemented with state machines. He was saying the high reliability of the system is mainly attributed to the requirement that systems be designed as statecharts first, which forces the designs to remain simple and analyzable.

<comment><USER4> > the high reliability of the system is mainly attributed to the requirement > that systems be designed as statecharts first, which forces the designs to > remain simple and analyzable.

This can also be due to the fact that statecharts inherently require every transition and event to be mapped, which is more up-front work than most programmers do on their state machines. Need a feature that requires a new state? Instead of glomming it onto the code (and, in a complex codebase, probably missing one transition or another), you have to do the work of putting it into the state diagram and handling every transition into and out of that state (from every one of the connecting states in the graph). From there, you have a much more completely accounted list of the changes that have to be made to other states, in addition to a more complete list for testing the resulting code.

> dealing with a complex system as a statechart is potentially harder than > dealing with code.

Yes, but it strikes me as harder in the same way that it's more difficult to get code to build when you have a strict linter and a strict static analysis tool in the pipeline; when reliability is worth a bit of developer- convenience-cost, we add procedures such that making changes takes more work at the beginning.

</comment></comment></comment><comment><USER5> For his logical combination problem, I think the solution is to use enums:

       State nextState(bool a, bool b, bool c, bool d)
       {
          enum A { A_NO = 0, A_YES = 1 } As[] = { A_NO, A_YES };
          enum B { B_NO = 0, B_YES = 2 } Bs[] = { B_NO, B_YES };
          enum C { C_NO = 0, C_YES = 4 } Cs[] = { C_NO, C_YES };
          enum D { D_NO = 0, D_YES = 8 } Ds[] = { D_NO, D_YES };
          switch (As[a]) + Bs[b] + Cs[c] + Ds[d])
          {
             case A_NO  + B_NO  + C_NO  + D_NO : return X; // 0
             case A_NO  + B_NO  + C_NO  + D_YES: return X; // 1
             case A_NO  + B_NO  + C_YES + D_NO : return X; // 2
             case A_NO  + B_NO  + C_YES + D_YES: return X; // 3
             case A_NO  + B_YES + C_NO  + D_NO : return X; // 4
             case A_NO  + B_YES + C_NO  + D_YES: return X; // 5
             case A_NO  + B_YES + C_YES + D_NO : return X; // 6
             case A_NO  + B_YES + C_YES + D_YES: return X; // 7
             case A_YES + B_NO  + C_NO  + D_NO : return X; // 8
             case A_YES + B_NO  + C_NO  + D_YES: return X; // 9
             case A_YES + B_NO  + C_YES + D_NO : return X; // 10
             case A_YES + B_NO  + C_YES + D_YES: return X; // 11
             case A_YES + B_YES + C_NO  + D_NO : return X; // 12
             case A_YES + B_YES + C_NO  + D_YES: return X; // 13
             case A_YES + B_YES + C_YES + D_NO : return X; // 14
             case A_YES + B_YES + C_YES + D_YES: return X; // 15
          }
       }

Let the compiler reorgonise the switch for efficiency. Do not: skip combination. Do not: reorder. Do not: remove the number that help see we got all cases. The compiler will error out if a case is duplicated. I wish we could make it detect missing cases too!

</comment><comment><USER6> If you're not willing to develop all the boilerplate code to encode your states, transitions, etc. Boost has two libraries to offer : Boost Statechart and Meta State Machine. One of my favorite question on Stack Overflow compares both: [https://stackoverflow.com/questions/4275602/boost- statechart...](https://stackoverflow.com/questions/4275602/boost-statechart- vs-meta-state-machine).

As with most Boost libraries, you're in for some serious headache if you decide to use them (or is it just me?) but it's very rewarding in the end.

<comment><USER7> Boost.SML (which is experimental) has garnered a lot of attention lately as a modern take on C++ state machines that can replace both of the options you raised.

[https://boost-experimental.github.io/sml/](https://boost- experimental.github.io/sml/)

Unfortunately I haven't seen any compelling real world usage of any of these libraries, and the documentation for SML leaves a lot to be desired.

<comment><USER8> I have been using SML in production for a little over a year now without issues, mostly for protocol machines.

I agree the documentation is not great though, you kind of have to go over the example code and try to figure things out on your own.

Using embedded data for keeping state (e.g. counters, etc; see [https://boost- experimental.github.io/sml/examples/index.html...](https://boost- experimental.github.io/sml/examples/index.html#data)) along with injecting std::functions to use as actions is pretty powerful.

Here's a basic example I wrote some months ago showing a hierarchical machine ([https://gist.github.com/indiosmo/08ab24181770125d5a2448d27f6...](https://gist.github.com/indiosmo/08ab24181770125d5a2448d27f6ae99f)).

Also please note my usage is pretty basic as I usually have the same machine ported to C# and Python so I tend to use the smallest subset of functionality supported by all the libraries in each language.

<comment><USER9> I didn't find a good way to handle unexpected events with SML. I know there is supposed to be a special catch all event which can do that, but that's only for the case the event is not used in the entire FSM, which makes it pretty useless. Thinking of rolling my own based on std::variant now.

</comment></comment></comment></comment><comment><USER10> There's an excellent book (and associated open source project) on using state machines for reactive systems or embedded systems in general: "Practical UML Statecharts in C/C++, 2nd Edition: Event-Driven Programming for Embedded Systems" by Miro Samek. Here's a link to the site where you can download it from ([https://www.state-machine.com/psicc2](https://www.state- machine.com/psicc2)).

</comment><comment><USER11> Harel Statecharts are mentioned elsewhere in this thread, and I endorse them. IMO they're a really great way to model your design.

IIRC Quantum Leaps [Samek; also mentioned in this thread] is similar to statecharts.

I've used Rational Rhapsody for C++ and I think it's actually pretty good. But there be dragons there. UML alone is a tough sell to some engineering crowds, much less generating code from diagrams. Holy wars abounds.

While searching for related info to this post I discovered that W3 has a relevant specification [1]. But I suppose it makes sense, IBM is a player here and they love XML there.

[1] [https://www.w3.org/TR/scxml/](https://www.w3.org/TR/scxml/)

</comment><comment><USER12> State machines are at the core of event-driven programming, but I often find coroutines to be a better, cleaner alternative. See this HN thread: [https://news.ycombinator.com/item?id=7676691](https://news.ycombinator.com/item?id=7676691)

</comment><comment><USER13> I have worked with generative hierarchical state machine engines for years, mostly for medical devices. It's relatively easy to "roll your own" finite state machine, but it gets unwieldy quickly for anything other than the most basic problem. The two classic approaches, tables and nested switch statements are hard to read. You need a drawing, and one from which executable code is generated.

Hierarchical state machine drawings are the way to go but it's generally not practical to build one on your own. That said, I wrote my own anyway: OOSMOS, the Object-Oriented State Machine Operating System ([https://www.oosmos.com](https://www.oosmos.com)). It generates object oriented C from a state chart drawn with the open source drawing tool UMLet. OOSMOS is open source.

</comment><comment><USER14> It sounds like he just re-discovered that a Moore machine is simpler to implement than a Mealy machine.

</comment><comment><USER15> Well another OO way would be to add a state member to the Heroine class and create a state class for every possible state (X,Y,Z). Each state can then be responsible for interpreting the input and transitioning to the next appropriate state (if any)

<comment><USER16> How do you manage combinatorial explosion as states increase?

<comment><USER15> The proposed state table is hardly more scalable. It's obviously not infinitely scalable but In my proposed way, you only have to specify the cases where a state transition actually occurs.

<comment><USER16> What you describe feels more like case classes.

</comment></comment></comment></comment><comment><USER17> Every computer, and every program, is a state machine. We invented machine language, and increasingly higher-level languages, specifically to provide a better way to program than making raw state machines.

When you find yourself using a high-level language to emulate, badly, a below- machine-language-level state machine, you should recognize that you are in a state of sin, and ask where you have gone wrong in life.

Thus spake the Voice of Experience.

<comment><USER18> I don't agree at all, and your baseless assertions misrepresent both history and technical requirements.

Contrary to your baseless claims, programming languages were not developed to avoid writing state machines. They were developed to provide programmers with convenient abtractions that made them more productive and helped them implement what they wanted to implement. Back in the real world state machines, either generated from a DSL or hand-rolled by developers, are still the standard solution to implement a lot of mundane components such as network protocols, language parsers, user interfaces, and even control systems. We're talking about applications where the state changes depending on inputs in spite of where the control flow might be. If your experience hasn't taught you that then you should not be paying attention to its voice.</comment></comment>


Ask HN: review my app - a minimalist's to-do organizer - <AUTHOR> http://www.getcodo.com/ <comment><USER2> I like the idea, but anything like this, at least for me, needs to be syncing to the web and have a web UI too. You don't necessarily need a mobile client, but a mobile optimized website for it is kind of a must have too.

But I agree that there's a need, though it is a crowded market.

<comment><AUTHOR> Yes, absolutely, the web syncing is a must. The idea is to take it a step further with some cool teamwork capabilities.

About web UI - there are a lot of web-based to-do list out there, so it's not really a big differentiator. For me, being a heavy user, web based to-dos didn't work very well and that was one of the main reasons I built Codo. Nevertheless I can see how it adds some value.

Mobile client - this one I'm also lacking most of all, so will do something about it sooner than later.

</comment></comment><comment><USER3> todo list apps are incredibly hard to get right. I have found I don't like most of them, and I have resorted to using tadalist most of the time. So far I am liking yours though.

It's really clean and simple, which is great. I also like that you used Air, even if others disagree.

I wish I could see all tasks at once. It looks like just adding an "All" list would accomplish that.

I also wish I could select more than one list at once (with cmnd and/or shift) and see both of them merged together in the main pane.

Adding tags with the pound sign is great, and the auto completion is even better. I really like that.

I wish the tag section could include a "tagless" option, to show me all tasks that lack tags.

If I select more than one task, I wish clicking a checkbox would effectively click all their checkboxes.

It really needs to have an online counterpart though. This unfortunately is really essential. I need my todo lists at home, at work, on my phone, just about anywhere. If you had that, I think I would totally use your app.

<comment><USER4> I also agree about using Air. I've only used Tweetdeck and this app on it, but it's got me interested in developing on it.

I understand there maybe be certain issues with the platform but it doesn't come without it's strengths.

</comment></comment><comment><USER5> It looks like you've already written the app. Most people won't install an app just to review it; however they will visit your site. You should have put the site together first and posted that. Then you'd have feedback before you started building it.

<comment><USER6> Good advice.

Minimum Viable Product: <http://en.wikipedia.org/wiki/Minimum_viable_product>

<comment><USER7> Minimum viable product of a desktop application is not a website describing what your application will do.

<comment><USER6> Actually, that's exactly what it is:

 _The canonical MVP strategy for a web application is to create a mock website for the product and purchase online advertising to direct traffic to the site. The mock website may consist of a marketing landing page with a link for more information or purchase. The link is not connected to a purchasing system, instead clicks are recorded and measure customer interest._

<comment><USER7> _The canonical MVP strategy for a web application is to create a mock website_ Note the _web application_ text.

Codo is a desktop application and therefor the MVP is a desktop application. Prototyping a web application for a desktop application usually requires a complete rewrite not iteration.

MVP in general seems suspect and something hackers should avoid.

Example: _Feature: (deploy first, code later) A link to a new feature in a web application may be provided in a prominent location on an existing website. The feature is not implemented, rather an apology, mock-up, or marketing page is provided. Clicks of the link are recorded and provide an indication as to the demand for the feature in the customer base._

<comment><USER6> I understand the difference in web vs. desktop. In this case, the intent as it relates to MVP is the same. The example given measured market interest without a prototype, so there is no iterative deployment issue.

I can understand the apprehension about MVP. As a coder myself, it is not an obvious approach (vs. putting out an actual beta) and I've not yet tried it. I'm interested in other people's stories and experiences though, and I'm not inclined to downmod a constructive conversation about it.

</comment></comment></comment></comment></comment></comment><comment><USER8> Using it! Love it so far, for the few minutes I've used. Perhaps this will replace my need for tadalist which I resort to every now and then when I have too much on my plate.

Would help to see numerical numbered list. I get a good kick personally knowing I have x things to get through.

<comment><AUTHOR> I like the numbered list idea as well. Thanks for a tip!

</comment></comment><comment><USER9> I didn't like that I had to download Adobe AIR for downloading and installing your app. What speaks against making a .dmg file and allowing the user to copy your app wherever she/he likes?

<comment><USER10> 1\. Is it even possible to run an AIR app from a local AIR engine, without installing AIR? That would be quite slick.

2\. Assuming that were possible, how is this to be packaged for Linux and Win32 such that it Just Works?

First time I looked at an AIR app I was also annoyed about having to install yet another VM thing. But, a few actually useful AIR apps later, I didn't care. So long as there isn't some creepy eternal background process running ( _cough_ Google updater _cough_ Apple's itunes spy _cough_ ) as residue, it's not an issue.

</comment></comment><comment><USER11> Nice app, but I'm not switching from my current scheme of a .txt file for local todos and Google docs for stuff I need to share or get from afar.

I wonder what it would take? I often have a list or two going, but .txt is just so convenient. Why leave emacs at all?

<comment><USER12> I have tried many methods and apps but always return to scraps of paper.

Current site I'm using is:

<http://teuxdeux.com>

Figured a few people might be interested in taking a look at this approach which is different to most others I've tried and quite simple.

</comment></comment><comment><USER13> As such I like it and it seems to be exactly the sort of thing I've been looking for, however there needs to be sync feature and a way for me to view and update my todo list from a browser. Add that and I'm really interested.

</comment><comment><USER14> Offtopic: I thought that the market for to-do organizers is supersaturated at this points. Is it that there just aren't any good ones? What do you expect the marked for this thing to be?

Personally, I use a white board for lists of projects that need to be accomplished, and my e-mail inbox for the list of things that need to be accomplished soon. Not a perfect system, but at least I can glance at both quickly as I always have Thunderbird open and sit in front of my whiteboard.

<comment><AUTHOR> There is a bigger vision for new kind of teamwork web service and methodology. It came from my own experience while leading a product development team for several years. General idea is to have a very light / low friction way of planning and tracking teamwork.

Most of current tools in the market focus on projects rather than teams. My believe is that there is a need for both.

To-do app serves as logic starting point for this, kind of a core.

</comment></comment><comment><USER15> I use Evernote for todo management. For me, the essential feature is, that the todo application is the same as the knowledge management (or notetaking) application.

For example, the todo might be "Fix the bug #1234". In Evernote, this will be the name of the note, and in the body of the note, I copy the bug description, and write any other stuff I discovered during my work. I found that the applications which only maintains the lists of todos are not useful for me.

<comment><AUTHOR> There are some ideas in this direction as well. It makes sense to organize to- dos and notes in the same fashion.

I'm actually using Evernote myself, it's a good app, but still there are are a number of unnecessary features, and couple critical ones missing.

</comment></comment><comment><USER16> A couple of things I would like: \- Mark todos as recurring \- Reminders/Alarms \- I would prefer if we could add more to todo. Add todo, give it a title, then fill in the body. \- A "Done" button rather than a checkbox \- Give todo items a due date in addition to today tomorrow someday.

You say this is a minimalist app, but these are essential for me.

<comment> jim-greer I wouldn't add any of those things... everyone has different 'essential' features.

<comment><USER16> Sorry if it came out wrong, that's why I said essential "for me".

</comment></comment></comment><comment><USER17> Great for a minimalist app, but I have found a bug already. I created one todo for today, then created another and immediately clicked the Move to Someday button. The Today todos all vanished except for one I had created earlier. Clicking on New Todo restored the lost items.

</comment><comment><USER18> Good work man.

One suggestion: I think it would be nice if you could tag when a task is "done". At first it doesnt mean much but I can imagine 3-4 weeks into using this id have hundreds of tasks completed. It would be helpful to be able to see when stuff was completed.

<comment><AUTHOR> I'm intending to add a dashboard which would look kind of like a calendar with would show completed task on each day.

</comment></comment><comment><USER19> Instead of for minimalists, why not just say "...for You." The minimalists know who they are, and those who aren't sure yet may not be ready to jump into all of the associations.

</comment><comment><USER20> Anyone else here use Org-Mode (ala emacs?) Getting a text file sync'd is an rsync matter (but it does require 1 box with a public DNS name/static ip).

</comment><comment><USER21> Like it so far. Air doesn't bother me, I guess I'm an Adobe whore. In a weird way I prefer it for simple tools like this.

</comment><comment> jim-greer This looks good, but I'm pretty happy with TaskPaper, which hits a very similar niche and doesn't require Air.

</comment><comment><USER22> Logo design must be inspired by Hulu.

</comment><comment><USER23> I like it, very clean and simple. The # tagging allows to make it flexible.

</comment><comment><USER24> I installed it on Ubuntu and the menu/tool bar was missing.

<comment><AUTHOR> Have to admit - haven't tested it on Linux yet. Will look into this today.

</comment></comment><comment><USER25> one thing you might like to look into is having it talk with some of the todo websites out there, like remember the milk (one of the more popular ones)

<comment><AUTHOR> Speaking openly, it's a business related issue.

The desktop app is free and it always will be. I'm even intending to open source it one day once the code is solid.

The business model is to provide added value via web-services. Some of it may also be free and some may be charged for.

So, being said that, It makes sense to provide migration path from RTM and alike, but integrating directly with it - not so much.

</comment></comment><comment><USER26> Maybe I'm dumb, but I can't quite figure out what the "Review" label means. What does it mean?

<comment><AUTHOR> It's kind of parking area where you can put to-dos which you haven't decided what to do with yet i.e. For example, for quickly dumping ideas etc. Later on you can go over the list and decide whether an item needs to be done today, someday etc.

On a related noted - some in-app "hints" that explains this would be great. Something I should really add ...</comment></comment>


Emails Reveal Why Steve Jobs and Phil Schiller Blocked In-App Kindle Purchases - <AUTHOR> https://www.macrumors.com/2020/07/31/emails-apple-blocked-kindle-purchases/ <comment><USER2> This article buries the lede, which should be at the top. It states:

> Yesterday, further emails published by the U.S. antitrust subcommittee > revealed that Apple offered Amazon lower App Store fees to convince it to > launch its Prime Video app on the App Store and Apple TV.

> The news that Apple made exceptions for Amazon for Kindle prior to 2011 and > for Prime Video has caused confusion among some observers, as Apple has > consistently claimed it treats every developer the same. Phil Schiller > stated ahead of the antitrust inquiry this week that all apps in the App > Store have been treated "the same - one set of rules for everybody, no > special deals, no special terms, no special code, everything applies to all > developers the same."

Phil Schiller was rather dishonest - implying that it always been this way, when it clearly has not. If his statement was identical to how MacRumors writes - the words "have been" are outside the direct quote from Schiller, where he could have said "are" in the present tense - then it was an outright lie. Not a good look in an era of renewed antitrust scrutiny.

</comment><comment><USER3> I find it funny that people keep quoting Steve Jobs as saying Apple didn't want to make money from the App Store. That may be what he said in public, but emails like this are evidence that he was not much different than the current crop of big tech CEOs.

</comment><comment><USER4> Here is my take on this:

Apple should demand its 30% cut...

... but ... companies should be allowed to compensate for their loss by adding 30% to the price when a purchase originates from the app store. They should also be allowed to inform the customer about this.

Seem overall pretty fair to me.

<comment><USER5> I think companies do that.

<comment><USER6> I hear YouTube plus does that, but they don't really label it as a fee or show a price comparison if bought through the web browser.

</comment></comment></comment><comment><USER7> Interesting to see the internal debate, even if i totally disagree with what they decided on. Has anyone ever tried to run a company will full transparency?</comment>


Show HN: Top Statistically High Causes of Death for Programmers (CDC/NIOSH) - <AUTHOR> http://sdrv.ms/13ekRCS

<comment><AUTHOR> Source: [http://www.cdc.gov/niosh/topics/surveillance/NOMS/icd-occ- fo...](http://www.cdc.gov/niosh/topics/surveillance/NOMS/icd-occ-form.html)

Data is '84-'98.</comment>


31 Years Later, "Star Castle" Finally Ported to Atari 2600 - <AUTHOR> http://starcastle2600.blogspot.com/p/star-castle-2600-story.html

<comment><USER2> Really impressive. Especially the in-cartridge LED's. It's a shame the magic of the clean vector lines of the original Cinematronics got a little lost in the port.

On that note; how fucking awesome were/are Cinematronic games.

<comment><USER3> _"Really impressive. Especially the in-cartridge LED's"_??

It is a cool idea, but I think that is the part I could have done. Programming this on the 2600 is a different story.

[Aside: I think those white LEDs are out of date; in those days, LEDs were red]

<comment><USER2> The LED's are programmable and _"actually flash with the gameplay"_. Yes that is really impressive.

</comment></comment></comment><comment><USER4> I'm not much of a gamer myself, but this is pretty impressive. I love the detail that went into making the cartridge / retro packaging. Pretty cool!

</comment><comment><USER5> It's impressive, but would it be worth investing in on Kickstarter? No.

<comment><USER6> That's for Lackluster users to decide.

<comment><USER6> oops, my phone spell "corrected" _Kickstarter_ to _Lackluster_. :)

</comment></comment></comment><comment><USER7> I loved Star Castle.

Man, this development effort was pretty impressive.</comment>


Angels Hate On Yuri Milner and Ron Conway's $150K to Y Combinator Start-Ups - <AUTHOR> http://blogs.forbes.com/nicoleperlroth/2011/01/31/angels-not-pleased-with-yuri-rons-150k-to-y-combinator-start-ups/

<comment><USER2> So it's news now that some businessmen don't like a decision made by another businessman/men?

Calling these investors angels is misleading because it makes it seem like angel investing is an act of altruism. These guys use this to imply that when something happens to make it more difficult for them to make money, it means it's bad for the startup environment in general. Get over it.</comment>


A New Provable Factoring Algorithm - <AUTHOR> http://rjlipton.wordpress.com/2014/10/18/a-new-provable-factoring-algorithm/ <comment><USER2> I'm a little confused as to what distinction is being made here between "provable" and "unprovable" factoring algorithms. Is it just that "provable" ones may output the correct result all the time, while "unprovable" ones are probabilistic and output the correct result with high probability? (E.g., roughly the distinction between P or ZPP on the one hand and BPP on the other hand, if these hypothetically ran in polynomial time and we were talking about decision problems.) Or is something even weaker meant by "unprovable", like retuning possibly incorrect results based not on a random input but on the number to be factored? I assume it's the former -- if it were the latter I'd say such things are not really factoring algorithms, practically useful as they might be -- but the way it's worded isn't exactly clear.

<comment><USER3> Well, in math some things are true but not provable unless our formal system is inconsistent. This is one of Gödel's incompleteness theorems.

It's therefore not far fetched that some algorithms might in fact always work correctly, but at the same time a proof for their correctness doesn't exist. You can never be sure (literally!) that you're not just confusing it with an algorithm that's wrong but happens to work on all problem instances you've thrown at it so far.

IIUC, when the article says unprovable, what it actually means is unproven. To correctly describe an algorithm as unprovable, you would have to have a proof of its unprovability, which would be quite something.

<comment><USER4> There's no obstacle in principle to proving that something can't be proved. As far as I understand things, the continuum hypothesis has been shown to be unprovable within ZFC (since ZFC is consistent with both the truth and the falsity of the continuum hypothesis, obviously it is unable to prove the hypothesis true or false), and theoretical computer science produces results of the form "to prove X, the following kinds of approaches cannot work".

<comment><USER5> "ZFC is consistent with …" must, of course, be replaced by "If ZFC is consistent, then so are …".

In fact, singling out ZFC gives you another example of this kind of result: if ZF is consistent, then so are ZF+C and ZF-C.

<comment><USER4> That all depends whether you're comfortable saying a contradiction is "consistent" with an unrelated proposition; I don't see much of a problem with it. Contradictions, being impossible, are logically consistent with everything else. I guess unprovable propositions _would_ switch over to being provable, though.

<comment><USER5> > That all depends whether you're comfortable saying a contradiction is > "consistent" with an unrelated proposition; I don't see much of a problem > with it. Contradictions, being impossible, are logically consistent with > everything else. I guess unprovable propositions would switch over to being > provable, though.

Oh, I see. I read "A is consistent with B" as "A+B is consistent", but I can see the validity of the reading "A+B is 'as consistent as' A".

</comment></comment></comment></comment></comment></comment><comment><USER6> I found an algorithm to determine the parity (even or odd) of the number of factors in N. Never wrote about it because IDK if it's unique, and also because it's time complexity seems poor and I never tried to prove any upper bound or make it faster. Anyone know of something similar? Would this be interesting?

<comment><USER7> A fast algorithm for parity would be interesting. There is currently no better way to compute parity than factoring N.

There is a well known hand wavy-argument that computing any function on the prime factors of arbitrary integers should be difficult. Very roughly the idea is that if you can compute such a function over the integers you can probably also compute it over other rings. Applying it over certain number fields would let you get information about the low bits of the prime factors which is thought to be hard.

There is a bit more information given in Terrence Tao's answer here [http://mathoverflow.net/questions/3820/how-hard-is-it-to- com...](http://mathoverflow.net/questions/3820/how-hard-is-it-to-compute-the- number-of-prime-factors-of-a-given-integer)

His answer is about counting the number of distinct prime factors but I think that it can extended to the parity of this value as well.

<comment><USER6> Thanks, the mathoverflow link was very informative. It's funny because I started out computing a number that I thought would be useful for factoring N, but it turned out that it would take one of 2 values depending if N was prime or had 2 factors (this is easily proven). Subsequent testing via software seemed to indicate it was actually determining the parity of the number of primes. I believe that's what it does in general but never bothered to go any further.

</comment></comment></comment><comment><USER8>
        I want to replace systems like AES with ones that uses
        the hardness of factoring for their security. Systems
        like AES rely on intuition and experimental testing for
        their security—there is not even a conditional proof that
        they are secure.

1) You can prove symmetric crypto is secure 2) In the light of (upcoming fast) factoring algorithms, using crypto that relies on factoring everywhere sounds very stupid.

<comment><USER9> Please point to a proof of the security of symmetric crypto if you can. Last I checked, such a proof would have still been considered an incredible breakthrough.

All that people are doing is to show that k rounds of cypher C can be broken, where actual implementations use K > k rounds, and then arguing that nobody has been able to break more despite a lot of effort, cypher C is probably secure in practice. That is good evidence, but not a proof.

<comment><USER10> One-time-use random data XOR your data = provably secure symmetric crypto, afaik. I could try to hunt down a proof of that one. _Impractical_ , sure, but provable.

Also afaik, others haven't been proven / can only be proven to have certain qualities, and leave gaps which have massive real-world implication. So yeah, I'm curious for other proofs too.

<comment><USER11> One time pads are currently the only provable "unbreakable" encryption. The proof is trivial.</comment></comment></comment></comment>


Haskell Primitives - <AUTHOR> https://www.fpcomplete.com/blog/2015/02/primitive-haskell <comment><AUTHOR> SPJ, co-author of GHC, added this awesome comment:

""Look at the implementation of other functions in GHC.Prim;they're all defined as let x = x in x."

This begs the question of why this strange code exists at all. Answer: the sole reason is to give Haddock documentation for the primops a place to live. GHC.Prim is processed by Haddock more or less like any other module; but is effectively ignored by GHC itself.

Worth saying this.

Simon"

<comment><USER2> I never understood why these weren't

        primop = let x = x in x
        (+#) = primop

or something else at least somewhat meaningful.

<comment><USER3> I don't know the answer to that, but I suspect it might be because there is some reason for the code in GHC.Prim to be monomorphic.

There is a Stack Overflow discussion about this that gives a bit more background, by the way. (The answer needlessly digressing into the lambda calculus is mine.)

0\. [http://stackoverflow.com/questions/15893524/what-is-the- mean...](http://stackoverflow.com/questions/15893524/what-is-the-meaning-of- let-x-x-in-x-and-data-float-in-ghc-prim-in-haskell)

</comment></comment></comment><comment><AUTHOR> This is one of my favorite innards-of-Haskell articles. It's such an intuitive way to look at how a super high level language like Haskell gets down to machine primitives. So important for optimization too.

I've been in HFT for a while and what strikes me is the performance optimizations you make in C++, Java, and Haskell are actually so so similar. Get down to the primitives. Avoid boxing. Avoid memory allocation at run time. Makes me think any language can be performant if you know what you're doing.

<comment><USER4> Exactly. And optimized Haskell doesn't quite look to "that beautiful FP language" anymore, but becomes more imperative.

<comment><AUTHOR> I was thinking more stream fusion and rewrite rules etc. GHC is a lot more powerful in that sense than the other language compilers I mentioned. A lot of it in Haskell just boils down to bang patterns on data types. And where it does get messy as you suggest (yeah it does), you can nicely abstract it away behind a pure library interface. That way the mess is encapsulated.

</comment></comment></comment><comment><USER5> "When GHC reaches a call to one of these primops, it automatically replaces it with the real implementation for you, which will be some assembly code, LLVM code, or something similar."

This, plus the "magic hashes" reminds me a great deal of Urbit's hinting syntax for jet propulsion (associating a body of high-level code in Hoon with an alternative (presumably low-level) implementation).

</comment><comment><USER6> Cool article! Very in-depth.

</comment><comment><USER7> Since Haskell is lazy, how does packaging io into a chain of function calls guarantee ordering? Does it depend on some primitives being strict?

<comment><USER2> There's an internal representation which ensure than IO actions are evaluated in the correct order.

</comment></comment><comment><USER8> Why this intentional misleading title?

<comment><USER9> What's misleading? It's an article about primitive operations and types in GHC Haskell.

<comment><USER2> The actual title is "Primitive Haskell" which certainly confused me when I first saw it (some weeks ago).</comment></comment></comment>


Battlefield robot (also) had security hole - <AUTHOR> http://www.boston.com/business/technology/articles/2009/12/19/battlefield_robot_had_security_hole/

<comment><USER2> Presuming that there's a post-mortem on this, it would be interesting to read if ever made public (quite doubtful). How many different processes had to either not exist or have failed for this to have happened? Surely, the military would require secured communications for such devices, wouldn't it? I presume that the manufacturer didn't outright lie to the military about the lack of encryption and that the military actively participated in the specification process. This, along with the drone video issue, is really quite strange.

</comment><comment><USER3> I never got a response to this question in the UAV thread, perhaps because it was so busy, perhaps because no one knows: Is there at least strong authentication crypto for the control protocols on these devices? They can carry weapons, after all.

<comment><USER4> I can't say I have any firsthand knowledge, but since the control inputs would be orders of magnitude less data than a video feed, my peace of mind and sanity has to assume so. I could be wrong, but please don't tell me.</comment></comment>


Building Android first could make it harder to raise money - <AUTHOR> http://sachin.posterous.com/android-gap

<comment><USER2> This is another good reason to build web/mobile-web/phonegap first if your product can be done (and almost everything can) with these technologies.

Not stirring the native/web pot, but web allows quicker iterations, no app store approval proceses, etc. etc.

At an introduction, does every investor even want to install another app? Do they install the app of every business that contacts them? I doubt it.

Lower the barrier to entry. I just built a demo app for a start-up who is looking for investment, and we had both camera/photo and location services in the browser without even needing phonegap. It's been viewed on iPhone, Android on both phones and tablets.</comment>


They Went Off the Grid. They Came Back to the Coronavirus - <AUTHOR> https://www.nytimes.com/2020/03/17/opinion/coronavirus-news.html <comment><USER2> Fun quote from the Grand Canyon river rafter: "Forgot to mention this, but my river name is One Chain. If it's at all appropriate to mention that, that would be tight."</comment>


People are dying of coronavirus because we're too slow at clinical research - <AUTHOR> https://www.statnews.com/2020/04/22/people-are-dying-from-coronavirus-because-were-not-fast-enough-at-clinical-research/ <comment><USER2> > _Obviously, for the patients who receive a placebo, this can be a sacrifice. > And it can be a hard one to make._

If the new idea is bad, the sacrifice is in the experimental group and the control group are the lucky ones that got the best treatment.

Note that in this cases where there is a "standard" procedure the control group receives the "standard" procedure. They are not left alone to die.

<comment><USER3> The problem with your theoretical argument is that we already know there is no standard treatment that works, and ventilators have a mortality rate of 66% to 90%.

So ... the control group is "left alone to die."

<comment><USER2> The 90% is probably wrong[1]. Anyway, the current method is saving 10%-34% of the patients. Nobody can guaranty that is perfect, but it's the best guess after years of treating patients with similar symptoms.

[1] What is the original source of the 90%? Is it from this article: [https://jamanetwork.com/journals/jama/fullarticle/2765184](https://jamanetwork.com/journals/jama/fullarticle/2765184) From the article:

> _The study included all sequentially hospitalized patients between March 1, > 2020, and April 4, 2020, inclusive of these dates._

> _As of April 4, 2020, for patients requiring mechanical ventilation (n = > 1151, 20.2%), 38 (3.3%) were discharged alive, 282 (24.5%) died, and 831 > (72.2%) remained in hospital._

If you ignore the 831 patients that were still in the hospital, and consider only the 38 that survived and the 282 that died, you get 282/(282+38)=88%. It is a very short time. What are the odds for people that survive more than 4 days in the hospital?

</comment></comment></comment><comment><USER4> We kinda do now know that remdesivir and chloroquine are useless. The trials are only continuing in a futile waste of research bandwidth to convince the sort of people that considers a wall in the desert more useful than a post office...

The rest is probably true, although maybe not as much now as in "normal" times? An intern with a clipboard and a quarter should currently be able to run a competent trial in NYC. I can't imagine approval being as difficult now as it usually is. Maybe there just aren't that many promising candidates? Maybe availability is a problem for more esoteric ideas?

From what I'm understanding, there is _a lot_ of experimentation of various levels of rigour (non-mortis) going on for changes in treatment protocols, such as timing, duration, and setting for ventilation, proning, fluid management, etc.

For every debate the public engages in regarding on the pro and cons of bleach-drinking, the medical profession is engaging in twenty debates on the best management of potassium levels to protect liver functions in diabetic patients etc

<comment><USER5> There is as active discussion happening between doctors on the front lines here and abroad. The world's healthcare practitioners are linked up through social media in a way that is truly inspiring. The best new ideas are coming from the bottom up.</comment></comment>


Ask HN: How to maximize income working as a SWE for 1 year? - <AUTHOR> tl;dr: if you were a mid-level software engineer looking for a new job, but you planned to work for only 1 year, what companies would maximize your income, and what would your expected income be for that 1 year?
Longer version:
In the fairly recent past I left my job as a software engineer to travel around and enjoy some time off. Due to the current pandemic, my travels have been paused since March and I don't expect them to return to full-form until at least summer 2021 (maybe borders will re-open towards the end of this summer and there'll be a short tourist season, but I'm not counting on crowded backpacker hostels, large cultural sites, or music festivals being the same this year). So, if I can't travel the way I like for the next year I figure I might as well get back on the saddle and try to maximize my income for the next 12 months.
So my question is: what are the best options as a SWE to make the most money in a year? And what is the expected income in that 1 year? A bit about me: some years as a software engineer at a well-known tech company and an undergrad degree from a top US university.
The most obvious answer to me seems to work at a FAANG, particularly Google, known to pay a high salary and without a 1-year cliff plus quarterly or even monthly RSU vesting, with an expected income in the $200-300k range. Start-ups might offer more equity, but I'm not planning to stick around long enough to make it worth it, and companies that backload their vesting too much (eg: Amazon) are also sub-optimal. Of course this is all assuming I can get the job in the first place, I just want to know what the best options are before I start applying. <comment><USER2> Fintech / HFT firms are known to be the highest paying for SWE, even higher than FAANG companies.

I routinely get LinkedIn messages and emails for companies like Citadel, Two Sigma, Hudson River, Jane Street - new grads for these companies make around $250-300k, and I imagine you would make a good deal more if you can negotiate well.

<comment><USER3> Damn!

I'm a mid-level with a MSIS working for a large financial company and I make less than $100k.

</comment></comment><comment><USER4> How old are you, and how long have you been traveling?

Seems like FAANG is a good bet. You could also go for a top tier hedge fund and bust your balls for a year. They pay straight cash and a huge bonus. Prob the best bet if you have a 1 year time limit, since the FAANG vesting doesn't help.

I'm well-traveled but want to do the same thing. Idk if I'm good enough for the top tier companies though. Might just chill at my fairly easy banking job at 130k.

<comment><USER3> $130k sounds nice. I don't even make that.

One word of caution about bonuses. I would not bet on a bonus. Bonuses tend to be very political. I have been screwed a few times when it comes to bonuses.

<comment><USER4> Yeah I just got screwed and got zero bonus this past year. Apparently the recruiter told me this. She basically did a play on words.

Where are you at though? This is 130k in nyc which is super expensive.

<comment><USER3> Ah, less expensive here (Philly). Still, it's like a 50% increase over what I make.

<comment><USER4> Philly is super chill. My cousin took a small pay cut (same company) to move there. Pays $650 for a nice bedroom in 3b/3ba. Used to pay $1850 in manhattan.

</comment></comment></comment></comment></comment><comment><USER5> Work for a remote company. Live in a low-cost place.</comment>


Memory and Brain Amyloid and Tau Effects - <AUTHOR> https://www.sciencedirect.com/science/article/pii/S1064748117305110 <comment><USER2> "(Theracurmin® containing 90 mg of curcumin twice daily)"

Another recent paper studied the beneficial effects of vanillin, which is a curcumin metabolite.

Contrast with: "Turmeric seems to have zero medicinal properties" on qz.com: [https://news.ycombinator.com/item?id=13517415](https://news.ycombinator.com/item?id=13517415)</comment>


Anjuna releases a free unseal tool for Hashicorp Vault - <AUTHOR> http://docs.anjunasecurity.com/vault-unseal <comment><USER2> Since they buried the lede: This is using SGX to hold the Vault master key. Unfortunately SGX still isn't available in servers.

<comment><AUTHOR> Intel SGX is currently available in Xeon E3 and Xeon 2100 series processors which are available on Microsoft Azure, Alibaba Cloud, Packet.com, Vultr.com, OVH Cloud, and more providers.</comment></comment>


Ask HN: Working in a startup - <AUTHOR>

I have recently graduated from college and taken up a job as a frontend developer in a startup. The job itself is fulfilling but I do find that I am missing out on something in life. I have learned lots of new things in the past 3 months and had the opportunity to work with great people on one hand but find that the other things that I like to do (blogging , participating in meetups and doing exercise) have taken a back seat due to the sheer amount of time and energy required for working. Any advice on how to cope with this <comment><USER2> Stick with the company for at least a year. You'll learn a lot of valuable skills. Staying with any one employer for at least a year makes it more likely you'll get hired somewhere else. Also everyone wants to know how many years, not months of experience you have with each skill you claim to have.

Be advised that it is quite uncommon for startups to actually succeed. The chances are quite good that your stock options - if any - will not be worth anything at all.

Do the very best you can to stay connected - or GET connected - with a professional network.

Take a long, hard look at whether your startup really will succeed. If you don't think it will, start sending out resumes towards the end of your first year on the job.

If you really do believe it will succeed, then decide for yourself whether it is worth all the effort and sacrifice.

Find some older, more experienced industry people who can give you some insight as to whether your startup will succeed.

Leave the office at least once per day, preferably twice, to go get a coffee or a smoothie. Do this on your own sometimes, so you can think, or invite a colleague. If you never leave the office during such a hard workday, you will burn out for sure.

Say you're having trouble on some code. Ask your boss out for coffee then discuss your problem with him. At Working Software, my boss and I did that every single day for three years.

<comment><AUTHOR> Thanks for this advice Michael</comment></comment>


Warren Buffett: "Really Successful People Say No to Almost Everything" - <AUTHOR> https://medium.com/accelerated-intelligence/warren-buffett-really-successful-people-say-no-to-almost-everything-ab78832ffebc <comment><USER2> > Buffett is very clear on what data he needs to know in order to make an > investment.

This is important lesson to even non-professional investor like me.

You should not use the data that is given or easily available to you just because it's easy to get. You should think first what you need to know and then get that data. Just because it's everywhere, does not mean it's valuable. In fact, it's usually the opposite.

Examples:

* Everybody pushes you information of how much their whatever has gained in 1,3,5 years compared to stock index. But that's too short period for stocks. It's usually not relevant information.

* Gains from bottom to bottom or top to top in stock market over market cycles is something you might want, not from bottom to the current highs as is usually used in marketing.

* Price index is everywhere but if you save for long term, you actually want to see what is the total return index. If you want to compare Berkshire Hathaway to SP500 you want to compare to SP500TR, not to the price index (BRK does not pay dividends).

* Taxes, costs, third party risk etc. must be counted in. Low cost index fund may have high hidden drags baked in because in some countries tax dividends paid to foreign firms differently from locals.

</comment><comment><USER3> No. _Really_ successful people aren't those who merely turned money into more money. (I also said _No_ to reading the article)

Chesterton says it a lot better than I could: [http://www.gutenberg.org/files/11505/11505-h/11505-h.htm#THE...](http://www.gutenberg.org/files/11505/11505-h/11505-h.htm#THE_FALLACY_OF_SUCCESS)

</comment><comment><USER4> >Buffett Strategy #6: Avoid the technology bandwagon

That's not true. Buffett bought a lot of Apple stock in 2016.

<comment><USER5> From my understanding Buffett doesn't invest in Apple because of a certain technology but because he sees them as a lifestyle brand with a sticky ecosystem.</comment></comment>


Geographic visualization with R's ggmap - <AUTHOR> http://blog.dominodatalab.com/geographic-visualization-with-rs-ggmaps/

<comment><USER2> This is gorgeous. I was recently asked if R could do mapping similar to the new maps in Tableau, now I can show it.</comment>


Ask HN: What's it going to take to stop all these awful non-popup popups? - <AUTHOR> Seems every time I click to read a story sourced through a Facebook post, there's so much garbage in the way of reading the content that it's not worth it to stay to read it. I end up leaving after closing sometimes as many as 3 of them if I'm really interested, usually less.<p>Maybe my detest is reflected in their analytics, but I imagine mine is a drop in a vast bucket of reader behavior. It seems like a hopeless struggle to sacrifice my ability to consume what interests me to try to send a message that this sort of end-user disrespect is unacceptable. <comment><USER2> When it comes to Facebook and stories, I have resorted to the comments section. Let someone else waste their time dealing with the popups, ads, and pagination. If there is not a useful comment that tells you that you _should_ read the article, or provides some insight into it, than don't bother at all.

It saves a lot of time and you probably missed nothing anyway. There are plenty of those articles that are like, "Find out..." or "She performed this miracle that saved her life, find out what it was!" with 10 pages of pagination and you read a comment that says: "I scrolled through it all and that information wasn't even in there."

Reminds me of what the television news always does: "You've been eating this at dinnertime for years and its actually poison! Find out what it is [after your dinner] at 6 PM!"

Honestly, websites with pagination are not websites I bother to read through unless they offer a single page view, or they have a maximum of two pages. Anymore than that and its just horrible web design that does not deserve the time of day.

The reason these websites continue to exist is because 1) they specialize in clickbait 2) they put up Google Ads or some system that helps them make a ton of money on those views -- only resulting in pennies, good enough for them; 3) but they advertise using Facebook ads or 4) coming up with clever headlines that make you want to click and 5) people keep clicking and reading, feeding their habits, and the websites are not penalized at all by Google or Facebook.

I know Google has started to penalize for popups on mobile, but as far as their ability to get past adblockers and still popup on desktops -- no biggie to Google.

So what reason does Google or Facebook have to really stop these types of website? Facebook is starting to initiate "fake news" but these websites that deal with the 5-20 pages of pagination? They aren't classified as doing anything wrong by Facebook or Google.

</comment><comment><USER3> Level up noob. Start using a browser that supports javascript blocking plugins.</comment>


Why I Do Not Support a Node Foundation - <AUTHOR> http://hueniverse.com/2015/02/24/why-i-do-not-support-a-node-foundation/ <comment><USER2> Maybe foundation should simply ensure non-enforcement of related IP.

<comment><USER3> But what value does the overhead of having the foundation and all its associated risks (such as those highlighted in the article) have over the current scenario, when many marks are happily going along day-to-day?</comment></comment>


No Girls, Blacks, or Hispanics Take AP Computer Science Exam in Some States - <AUTHOR> http://blogs.edweek.org/edweek/curriculum/2014/01/girls_african_americans_and_hi.html

<comment><USER2> This is a strange article. It leads with a somewhat incomplete headline and misleading intro paragraph, but eventually "comes clean" with a bit more explanation.

Yes, Wyoming had no female, hispanic, or African-American test takers. It also had no test takers of any kind.

Mississippi had only one test taker. Yet, according to the article, Ericson was "surprised" that none were African-American since African-Americans make up 37 percent of the population (still making them a minority). She was apparently not surprised that the White majority was not represented in the tests. The sole test taker was hispanic.

Montana also had no hispanic or African-American test takers. Yet, it only had 11 test takers and stastically one would expect 0 hispanic and African- American test takers given the demography of Montana.

Basically everywhere the exam was taken in reasonable numbers, females were respresnted, but underrepresented in terms of demographics. Ericson does not specificy if the numbers had grown from previous years. Interestingly, many were from girl-only schools. (To me this suggests in the absence of certain social pressures, girls are more likely to take computer science courses.)

And unsurprisingly, in places where relatively few hispanics live (at least in terms of offical demographics), few or no hispanics took the test.

What I find disappointing about the article is that by excluding "interesting facts" about white male test takers, we don't really know the significance of the passing rates of the hispanic test takers (although we do learn that African-Americans had the lowest pass rates).

The "interesting" thing about Mississippi is that _Mississippians_ are underrepresented, regardless of race or color. And I'd go so far as to say that's really the only interesting thing for _any_ of the states which had fewer than 100 people take the test.</comment>


HN Show: ConcussionJS, an experimental, rapid web dev platform - <AUTHOR> I recently created an experimental, rapid web development platform called ConcussionJS. Its initial target use case is converting HTML prototypes into fully functional applications without the hassle of traditional web development.
Six steps to test ConcussionJS for yourself: 1) Create your HTML markup 2) Include the ConcussionJS JavaScript library (http://www.concussionjs.com/concussion.js) 3) Add additional ConcussionJS HTML attributes to your markup (Built on KnockoutJS syntax; example at https://github.com/concussionjs/concussionjs-core) 4) Load your HTML file in a browser (Chrome and Safari tested) 5) Go to admin console add some records 6) Re-load your HTML file
Voila -- you now have a dynamic, database backed website. No custom client-side script required, no server-side coding or hosting, and you can try ConcussionJS on an HTML file located anywhere, even running off the file system directly (e.g., file(s) could reside on your desktop sans web server).
ConcussionJS can be used both as a cloud platform or run locally. Find more deployment option details at https://github.com/concussionjs/concussionjs-core
Tutorial: http://www.concussionjs.com
Example Applications:
+ http://samples.concussionjs.com/contacts.html + http://samples.concussionjs.com/twitter-bootstrap/blog/ + http://samples.concussionjs.com/foundation/timesheet/
ConcussionJS JavaScript library: http://www.concussionjs.com/concussion.js
ConcussionJS mission description: http://www.concussionjs.com/#about
Github page: https://github.com/concussionjs/concussionjs-core
I'm looking for feedback and would welcome ConcussionJS collaborators!! <comment><AUTHOR> Thanks everyone for checking out the site. Would be great to get your perspectives on any of the following:(i) Do you think I should add a custom ConcussionJS sign up/login, or stick with Google/Facebook authentication?, (ii) what kind of projects would you use something like this for, (iii) what additional features would you like to see?

<comment><USER2> Personally, I'd love to see a simpler setup process. Maybe allow using just MongoDB instead of both Redis and MongoDB?

It was also a bit annoying (to me) that at the beginning of installing from source it seemed possible to install somewhere other than $HOME but the install.sh scripts had so many references to $HOME I decided it wasnt worth changing them all.

<comment><AUTHOR> Great feedback, and agreed on both fronts! I added Redis to the mix because it has support for a really nice rate limiter pattern that's simple to implement, but its probably overkill for what people want when they deploy locally for experimentation or to contribute to ConcussionJS. I should probably have a production fork of concussionjs-proxy for the short term with the rate limiter, and a master fork built exclusively on mongo.

RE the built in dependencies to the $HOME directory, I agree that it's a weakness of the install scripts. I'll add this as an issue at [https://github.com/concussionjs/concussionjs- core/issues](https://github.com/concussionjs/concussionjs-core/issues).

BTW, there is a command-line tool that I've yet to document that you might find handy. It supports:

$ cjs app --create YOUR_APP_NAME $ cjs app --start YOUR_APP_NAME $ cjs app --stop YOUR_APP_NAME $ cjs app --restart YOUR_APP_NAME

You can see the documentation by typing: $ cjs

\--create also supports a --template argument (e.g., cjs app --create YOUR_APP_NAME --template YOUR_TEMPLATE_NAME) which copies the contents of the specified folder with the matching name in concussionjs-core/app_template.

Behind the scenes the command line uses a cool util called mon ([https://github.com/visionmedia/mon](https://github.com/visionmedia/mon)), a super lightweight version of Monit, to keep the nodejs process running for the app specified (e.g., monitoring the process and restarting as necessary).

<comment><USER2> Thanks - sounds good.

<comment><AUTHOR> Fixed the $HOME directory dependency issue! Thanks again for the feedback.

</comment></comment></comment></comment></comment><comment><USER3> Clickable

[http://www.concussionjs.com](http://www.concussionjs.com)

</comment><comment><USER2> This looks rather nifty!

</comment><comment><USER4> Amazing project. Change the name immediately. Concussion is a horrible name...

<comment><AUTHOR> What do you think about MomentumJS? I have that domain registered, too.

<comment><USER5> I also agree, not only was remembering concussionjs hard, but it's also too long and confusing. The project is really really awesome though!!

I'd suggest to implement an abstraction allow implementation of more flexible models by using: [http://en.wikipedia.org/wiki/Domain- driven_design](http://en.wikipedia.org/wiki/Domain-driven_design)</comment></comment></comment>


Kivy - Open source cross-platform library for rapid development - <AUTHOR> http://kivy.org/

<comment><USER2> Why on earth do they shy away from saying the word "Python" anywhere? Seems pretty relevant, considering it's a Python library. It's a pretty important fact for potential users, and one they would probably want to know sooner rather than later.

Personally, any sense of excitement and discovery I had upon visiting the front page was killed pretty flat when I spent a couple of minutes skimming the front page for _what_ it's a library _for_ , visiting 'About' and 'Docs', returning to the front page for a closer look, and finally figuring it out by taking a look at the sample source and noticing the reference to Cython.

The Github readme is much denser, and I think a lot of people would prefer this intro:

> Kivy is a Python library for development of multi-touch enabled media rich > applications. The aim is to allow for quick and easy interaction design and > rapid prototyping, while making your code reusable and deployable.

> Kivy is written in Python, based on OpenGL and supports different input > devices such as: Mouse, Dual Mouse, TUIO, WiiMote, WM_TOUCH, HIDtouch, > Apple's products and so on.

<comment><USER3> I had to go to the Wikipedia page for Kivy

<https://en.wikipedia.org/wiki/Kivy>

to find out that the "Kivy language" is QML! Seems to me that should have been spelt out a bit more clearly too.

It looks great!

<comment><USER4> No, it's not QML. It's similar in the concept, but that's all :) Look at our examples, you'll see how is it.

</comment></comment></comment><comment><AUTHOR> Kivy is running on Linux, Windows, MacOSX, Android and IOS. You can run the same code on all supported platforms. Many parts are written in C using Cython and tested with regression tests.

This is awesome!

Kivy uses natively most inputs protocols and devices like WM_Touch, WM_Pen, Mac OS X Trackpad and Magic Mouse, Mtdev, Linux Kernel HID, TUIO. A multi- touch mouse simulator is included. The graphics engine is built over OpenGL ES 2, using modern and fast way of doing graphics. The toolkit is coming with more than 20 widgets designed to be extensible.

Kivy Github page - <https://github.com/kivy/kivy-website/>

<comment><USER4> That was the website github, the code is at <https://github.com/kivy/kivy>

</comment></comment><comment><USER5> Well, this addresses atleast some of the issues regarding the future of python that someone else was worrying about a few days ago on HN. Kivy promises to be fast (which is usually the main sourpoint for the be-all do-alls)

""" Kivy is fast. This applies to both: application development and application execution speeds. We have optimized Kivy in many ways. We implement time-critical functionality on the C level to leverage the power of existing compilers. More importantly, we also use intelligent algorithms to minimize costly operations. Most importantly, we use the GPU wherever it makes sense in our context. The computational power of today's graphics cards surpasses that of today's CPUs by far for some tasks and algorithms, especially drawing. That's why we try to let the GPU do as much of the work as possible, thus increasing performance considerably. """

Itching to try!

</comment><comment><USER6> It looks like this is a Python library. How can you use it to make an iOS app like the site claimes?

<comment><USER7> It uses cython, which means "Python compiled to C"

<http://cython.org/> <https://github.com/kivy/kivy-ios>

<comment><USER8> So the end to end build is not yet available out of the box?

<comment><USER9> There's a tutorial which looks slightly old: <http://kivy.org/docs/guide/packaging-ios.html>

If Kivy wants to become "the primary Python for iOS" rather than just "yet another cross-platform Python implementation which nobody uses for serious iOS projects", they have to nail packaging more convincingly.

</comment></comment></comment></comment><comment><USER10> This seems really neat .. I have met a few folks who don't share my enthusiasm for MOAI recently, on the basis of not being able to use much Python (I'm a huge MOAI fan and love Lua) so now there is a great option.

<comment><USER11> What is MOAI?

<comment><USER12> Well, it could be those giant statues on Easter Island ... or a quick Google search for "moai lua" turns up this: <http://getmoai.com>

<comment><USER10> Yes, sorry I didn't include that reference, that is indeed what I meant. I had sort of gotten used to folks on HN knowing what MOAI was when I mentioned it, silly me .. ;)

Anyway, MOAI is also great in this regard: cross-platform, single language development.

</comment></comment></comment></comment><comment><USER13> how would one start developing cross platform mobile apps with this? From what I have read, they only provide the GUI part? So the logic (GPS, Contacts etc) has to be handled by something else? Anyone knows?

<comment><USER14> I was wondering the same. Seems to me that it's mostly limited to games then.. Probably some interfaces for webapps which don't rely on phone APIs..

<comment><USER4> It's not limited to game, check the gallery :) It's funny because before we did the contest, peoples was thinking it's not made for games...

<comment><USER14> How good is twisted working? That should give nearly every internet protocol to the developer :D

Especially IMAP/POP/SMTP (+SSL)? :)

<comment> qua-non [http://kivy.org/docs/guide/other-frameworks.html#using- twist...](http://kivy.org/docs/guide/other-frameworks.html#using-twisted- inside-kivy)

</comment></comment></comment></comment></comment><comment><USER15> Interesting. Might be fun to play now that bigger touchscreens are cheap enough. Need to check how well they're supported under Linux nowadays. Used to be quite sketchy.</comment>


Show HN: Textdb.dev – simple data sharing for fun projects - <AUTHOR> https://textdb.dev <comment><AUTHOR> I put this together because I couldn't find a dead simple data sharing service for a side project. I didn't want to deal with auth, or tables, or anything complicated -- just a small amount of data that I could monitor by leaving open a window.

If you have suggestions, I'd love to hear them.</comment>


Why You Should Consider Ditching Your Car or Bike for an Electric Bike - <AUTHOR> https://slate.com/business/2018/02/e-bikes-could-be-the-next-big-thing.html <comment><USER2> I replaced mine with a 48v 1000w 12amh fat tire bike. Pretty satisfied with the decision, it's a lot faster and cheaper but I've only put 1000 miles on it in the past year.

</comment><comment><USER3> Which will still get you mowed down on unprotected bike lanes (or streets without protected bike lanes). And confiscated / fined in NYC. Oh well.</comment>


Show HN: Community ASCII Wall using Socket.io for Real Time Updates - <AUTHOR> http://blog.andrewmunsell.com/post/30012142975/wall-of-text-experiment-with-node-js

<comment><AUTHOR> I just built a wall of text app using Node.JS and Socket.io... It's just sort of an experiment and break for me, but let me know what you think.</comment>


Ask HN: Why does HN use a table instead of an ordered list? - <AUTHOR>

I am currently learning web development and was interested in their design choice. Also, if there are other sites you would recommend studying, please share with me! <comment><USER2> I would not recommend using this site as a good example to learn from.

If you're going to learn web design: 1) [http://teamtreehouse.com/](http://teamtreehouse.com/) 2) [https://developer.mozilla.org/en- US/docs/Web](https://developer.mozilla.org/en-US/docs/Web)

Keep away from [http://www.w3schools.com/](http://www.w3schools.com/) they information is not accurate, they only care about their ranking on google.</comment>


KeyDB: A Multithreaded Redis Fork - <AUTHOR> https://github.com/JohnSully/KeyDB <comment><USER2> What we are doing instead: just started a Redis Proxy project (designed by Fabio and me, coded by Fabio Nicotra, so we started yesterday). We will start abstracting a Redis Cluster this way so that you can talk with to it like it was a single instance, but this is not enough. We also are planning for:

1\. Tooling to make it simpler to start a Redis Cluster in a single host if all you want is to leverage all the cores.

2\. Smartness in the AOF rewriting so that instances on the same host will do a better use of the disk.

3\. A caching mode for Redis Cluster. This is useful for deployments where most accesses are idempotent and you have for instance a cluster of 10 physical hosts running, for instance, 32 Redis instances each. When an host is no longer reachable we don't want failover in that case, but just reassignment of hash slots to the other masters. Think at consistent hashing but on the server side.

Two years ago I had this private branch that I wrote in a few days that threaded Redis, with N event loops on N threads. After realizing that it worked and scaled, I considered how many bugs such Redis would have, how harder it would be to add features, and how incompatible is to have a single huge instance with Redis persistence method. In that moment and after consulting with other engineers I decided that for a few years our strategy will be Redis Cluster. Maybe I'll change idea in the future? Maybe, but first let's explore what can be done with a shared-nothing architecture that keeps the promise of continuing to deliver software that hardly crashes ever.

EDIT: Obviously the proxy itself is multithreaded :-D

EDIT2: Btw jdsully already sent a PR to the original Redis project, so I guess a smart developer exposed to the Redis core is great anyway!

<comment><USER3> The problem with clustering still becomes lower queries per GB as instances can't share data. Redis itself runs in RAM so storage is at a premium.

One of my main reasons for doing multithreading, and FLASH in the first place was to make Redis work well for much larger value sizes.

I really think we have different use cases in mind.

<comment><USER2> You mean the use case where you have a single very "hot" key? Yes, that's a limitation, but in that case replicas can scale read load very well. But indeed this is a special case where threads are a better model: yet this is a very niche use case that anyway ends being inherently unscalable because the object to manipulate is a single one, so accesses end being serialized in one way or the other.

I understand the point about Redis on Flash. Redis Labs is also doing a lot of things with this model. I believe it is super useful for some folks. Yet I believe what makes Redis "Redis" is that there are no tricks: it runs the same always whatever the access pattern is. I want to stress this part over the ability to work with bigger data sets. Moreover I believe that persistent RAM with huge amounts is in the horizont now.

<comment><USER4> > yet this is a very niche use case

I've always thought that one of the sensible basic data structures that Redis could support operation on is an (int->int) digraph.

There are a lot of things you can build on top of "digraphs as a toolkit object"—just see the lengths people go to to get PostGIS+pg_routing installed in Postgres, just to do some random digraph queries to their (non-geographic) RDBMS data.

IMHO digraphs very similar to Redis Streams in that sense: they're something that might seem like a single high-level "use-case" at first glance, but which is actually a fundamental primitive for many use-cases.

However, unlike a hashmap, list, set, etc., a digraph is necessarily—if you want graphwise computations to run efficiently—one large in-memory object with lots of things holding mutual references to one-another that can't just be updated concurrently in isolation. But, it is likely that people would demand multiple writers be able to write to disjoint "areas" of the graph in parallel, rather than having to linearize those writes just in case they end up touching overlapping parts of the graph data.

If I understand Redis Streams, it was created because there's no real need for things like Kafka to do their own data-structure persistence. Data-structure persistence is not the "comparative advantage" of a thing like Kafka; network message-passing with queuing semantics is. High-level infrastructure pieces like Kafka should rely on lower-level infrastructure pieces like Redis for their data persistence, and then they can _just_ do what they do best. Is that accurate?

If so, then I would argue the same thing applies to graph databases: persisting and operating on data-structures is not really their job; it's just something they do because there isn't anything to delegate that work to. Redis could be such a thing. But not if it's single-threaded.

This is, just from the top of my head, a use-case (set of use-cases?) that _needs_ threading (or at least "optimistically threaded, with fallback to linearization") for writes. And it's not really all that niche, is it? Not any more than message queues are niche.

<comment><USER5> It seems like RedisGraph would fill this need. It builds upon Redis to provide a graph database. [https://oss.redislabs.com/redisgraph/](https://oss.redislabs.com/redisgraph/)

</comment></comment></comment></comment></comment><comment><USER6> Redis being single-threaded has always been its single most important advantage in my opinion. It eliminates whole classes of possible bugs and makes the software much simpler.

<comment><USER7> Multi-threading is not some crazy new fangled idea. A huge portion of the software you use every day is multi-threaded. It's not _that_ hard to get right. I don't see why the redis maintainers don't have the skills to do it.

<comment><USER8> Retrofitting multi-threading into an existing design, is actually a pretty big challenge. Especially if you're going with shared everything multi-threading as is common in C. I won't name and shame, but I've started using a well known "internet scale" project that recently added multithreading, and have run into three major locking errors already.

These are smart people, they've written a great program, but they didn't make the right locking decisions.

The approach antirez mentioned in a sibling thread of a multithreaded proxy process that communicates to multiple single threaded instances sounds much easier to implement without ending up with data in the wrong places. Although, even that doesn't necessarily need to be multithreaded either. A master process doing healtchecking and sending status to the worker processes, that are using a eventloop would probably work just as well.

</comment></comment></comment><comment><USER9> Heh, I did this back in 2012: [http://thredis.org/](http://thredis.org/) (Not only is it threaded, it also supports SQL).

<comment><USER10> And Alibaba did it a couple years ago with Pedis! [https://github.com/fastio/pedis](https://github.com/fastio/pedis) (looks like the project is still active, but I don't really know of anyone else who uses it.)

</comment></comment><comment><USER11> For anyone wanting an overview of how KeyDB works, its at the bottom of the Readme.

"KeyDB works by running the normal Redis event loop on multiple threads. Network IO, and query parsing are done concurrently. Each connection is assigned a thread on accept(). Access to the core hash table is guarded by spinlock. Because the hashtable access is extremely fast this lock has low contention. Transactions hold the lock for the duration of the EXEC command. Modules work in concert with the GIL which is only acquired when all server threads are paused. This maintains the atomicity gurantees modules expect."

<comment><USER12> So it is still single threaded in the semantics that people expect (which is great news). And should also not expose users or the codebase to nasty threading bugs.

</comment></comment><comment><USER13> > In addition we wanted open source implementations of features currently only > available in proprietary modules. We feel a fork is the best way to > accelerate development in the areas of most interest to us.

This is probably the most important part of the fork.

<comment><USER3> In that vein we have FLASH support baked in. It works a bit differently than Redis Enterprise (uses File System snapshotting), but it's also a lot less complicated.

<comment><USER14> What tradeoffs have you made by doing so?

<comment><USER3> The trade off is the kernel decides what is paged into RAM and what stays on disk. Less control, but also less overhead. This is a similar approach to Squid.

<comment><USER14> I don't think users are going to be happy with that approach in the long term. It's well advised to control eviction behavior yourself. (Varnish learned this lesson ages ago, notwithstanding phk's original paper.)

InnoDB's buffer pool documentation is instructive: [https://dev.mysql.com/doc/refman/5.7/en/innodb-buffer- pool.h...](https://dev.mysql.com/doc/refman/5.7/en/innodb-buffer-pool.html)

<comment><USER4> If you're running a Redis instance as the only thing on the VM instance it's on (very likely in my experience), is there a reason to reimplement all this policy logic in Redis, rather than simply tuning the caching policy of the existing OS kernel to make it perform well for Redis (maybe at the expense of performing well for any other process you might run under that same kernel)?

<comment><USER14> In theory that makes sense, but I'm not aware of any mechanism that allows you to tune a kernel's VMM to suit an arbitrary workload/access pattern. Are you?

</comment></comment></comment></comment></comment></comment></comment><comment><USER15> This seems to keep the important bit about Redis single threading, that is that you have exclusive access to the db in a script or `multi`. That property allows for a lot of things that wouldn't be possible otherwise. Also leads to very predictable performance characteristics.

That said, Redis has always been faster than I've needed, and that's leaning on it pretty heavily, so I kind of agree with antirez that adding this complexity may not come with much benefit. Will be interesting to see where this goes.

<comment><USER16> According to the README all access to the data is effectively single threaded. Only the network work is actually multi-threaded.

> Access to the core hash table is guarded by spinlock. Because the hashtable > access is extremely fast this lock has low contention. Transactions hold the > lock for the duration of the EXEC command.

<comment><USER3> This will be improved in future versions. One of my major goals is to make KeyDB work well for large value sizes, >1MB. There's no reason you shouldn't be able to cache images as well as text.

FLASH support was the first step, and we've removed a bunch of unnecessary copies along the way but there is still more work to do.

<comment><USER9> FWIW - my notes on how locking was done in Thredis: [https://github.com/grisha/thredis/blob/master/README- THREDIS](https://github.com/grisha/thredis/blob/master/README-THREDIS)

<comment><USER3> Thanks! I did look at thredis originally, but the github seemed to not have been maintained at the time.

</comment></comment></comment></comment></comment><comment><USER17> Multithreading is great, getting data to walk across sockets on multi-socket machines can be a challenge. Source: have dual-socket E5-2670.

Multithreading eliminates 99.8% of snapshot overhead: [https://www.slideshare.net/secret/sLTnWqcPV7G0HK](https://www.slideshare.net/secret/sLTnWqcPV7G0HK)

Can also help reduce snapshot create and load time by 20-70% or so: [https://www.structd.com/](https://www.structd.com/)

But hey, I'm right behind you :)

</comment><comment><USER18> I find it really surprising that it took someone a month (I think the author stated this on Reddit) and yet redis has been so hesitant to do it. What are the downsides at this point?

<comment><USER19> antirez recently provided a full response [http://antirez.com/news/126](http://antirez.com/news/126)

<comment><USER20> He says most setups are network or memory bound, so I don't understand this reasoning:

"Well, that's the plan: I/O threading is not going to happen in Redis AFAIK, because after much consideration I think it's a lot of complexity without a good reason. Many Redis setups are network or memory bound actually. Additionally I really believe in a share-nothing setup, so the way I want to scale Redis is by improving the support for multiple Redis instances to be executed in the same host, especially via Redis Cluster."

The RAM would have to be a multiple of the dataset, because its shared- nothing, running right into one of the two common limits, right?

<comment><USER8> I would assume a cluster on a single node would partition the data, rather than replicate it. Without an in-depth knowledge of redis, I can't really predict if 4 instances sized for 1GB would have more overhead than 1 instance of 4GB, but I'm willing to hazard a guess that it would be fairly close. On the network side, it's a wash if you're already bound by the interface line rate.

</comment></comment></comment></comment><comment><USER21> If using `db-s3-object`, does KeyDB uses that as a seed on startup?

Eg: Running KeyDB on AWS, the EC2 instance fails and is replaced by another one running the same config. Will it read the last S3 dump on startup?

<comment><USER3> Not yet implemented. I'm hoping to get that fixed today.

I mostly use slaves for recovery with the database dumps as a last resort. So I overlooked that.

<comment><USER21> Amazing!

Do you have any special config to sync to replica before accepting incoming connections in a straightforward manner?

<comment><USER3> Yea we have external management software we wrote. We've been looking into releasing this as well, but it's not cleaned up yet.

<comment><USER21> Looking forward to that. In the meanwhile I'll wait and test the S3 option when it's live.

<comment><USER3> I just implemented S3 loading in this change: [https://github.com/JohnSully/KeyDB/commit/a45f212693956a6fb1...](https://github.com/JohnSully/KeyDB/commit/a45f212693956a6fb1aacf465d88e940bbbfd56f)

Its currently in the unstable branch, once it gets shaken out a bit I'll release 0.9.3 with it included.

</comment></comment></comment></comment></comment></comment><comment><USER22> How fast are writes compared to normal redis? Is this being used in production anywhere?

</comment><comment><USER23> Why scrolling on their website [https://eqalpha.com/](https://eqalpha.com/) is so broken?

</comment><comment><USER24> > Unlike most databases the core data structure is the fastest part of the > system. Most of the query time comes from parsing the REPL protocol and > copying data to/from the network.

That's the TL;DR; of "What tradeoffs are required?"

From what I can tell, they extracted an embedded redis DB and wrapped it in a multi-threaded protocol layer. So execution and behavior of Redis is identical, but all the "not actually a DB" parts are faster and more concurrent.

<comment><USER3> Author here. We added fine grain locking to the redis codebase. The goal is to expand what's concurrent over time.

Currently network IO and query parsing are multithreaded. Access to the core hash table is under lock. However network IO and query parsing take up most of the time in Redis queries. Hash tables are pretty fast. The goal is to add a reader/writer lock here in the future.

The lock itself is also interesting. We used a ticket lock which is a variant of a spin lock. This ensures fair locking and reduces the variance in latency dramatically. Standard POSIX locks are too slow.

<comment><USER16> I've always wondered what the best concurrency approach would be for a fully compatible fully concurrent Redis. The single-threaded nature makes it naturally expose some strict semantics to the user, which is tricky. What are you planning on doing to push past the global lock for the hashtable?

<comment><USER25> [https://news.ycombinator.com/item?id=19369794](https://news.ycombinator.com/item?id=19369794)

This is an example that is completely lock free.

</comment></comment></comment></comment><comment><USER26> Does anyone have a link to the discussions on why redis stayed single threaded?</comment>