How we were building the wrong product - <AUTHOR> http://kilometer.io/blog/how-we-were-building-the-wrong-product-for-the-last-10-months/?ref=news.ycombinator.com/newest <comment><USER2> The fact that they feel the need to have mixpanel, segment.io and hotjar on their webpage doesn't exactly show confidence in the product :p

Analytics is such a saturated market at this point. It will be interesting to see how things shake out in terms of consolidation / niches / price ranges.

It is also interesting to me that they decided to invent yet another format in which customer data must be exposed. Just like the previously mentioned companies. [https://www.w3.org/2013/12/ceddl-201312.pdf](https://www.w3.org/2013/12/ceddl-201312.pdf) (shameless plug: [https://github.com/mkohlmyr/ddl-json- schema](https://github.com/mkohlmyr/ddl-json-schema))

<comment><USER3> I saw it mentioned while doing searches a while back and ended up on Quora where a Kilometer rep says in his post:

> our users often call us "Mixpanel _on steroids_ " [1]

I then click through to find ironically, that Kilometer themselves use Mixpanel..

[1]: Screenshot: [http://thumbsnap.com/i/kFfmeZEx.png?0310](http://thumbsnap.com/i/kFfmeZEx.png?0310) (from [https://www.quora.com/What-are-the-best-Mixpanel- alternative...](https://www.quora.com/What-are-the-best-Mixpanel- alternatives))

</comment></comment><comment><USER4> Will their next post be "How we were building something that already existed for the last 10 months"? Because this a lot like [https://www.geckoboard.com/](https://www.geckoboard.com/)

<comment><USER5> And as far as I can tell, there are around 20 companies offering exactly the same thing in this space - most of which have been around for a couple of years already.

Not sure it's a good idea to pivot into this space.

</comment></comment><comment><USER6> Overall I think analytics aren't as good a market at they sound.

The basic problem is that they are something you bolt onto your business after the fact rather than something that is core to running a business. They are a "nice to have" rather than "have to have" and when something new comes along it is too easy to replace them.

<comment><USER7> It's the difference between a vitamin and a pain-killer.

You want to make pain-killers, because people _need_ them.

Vitamins can be helpful. But people don't usually need them.

<comment><USER8> I would have put money on this example being structured the other way around. People can live with pain; they can't live without vitamins. They only don't need vitamins in the same sense that people don't need oxygen.

<comment><USER9> Pretty sure that the post you're responding to meant "vitamin supplements."

<comment><USER7> That's it, thanks.

</comment></comment></comment></comment></comment><comment><USER10> Not being familiar with this product to begin with, I'm wondering if he gave up on his product too quickly!? In my experience customers are typically wrong and misinformed - the whole "faster horse" concept at play. Instead of acquiescing to his customers whims (I want what I want when I want it), could he have had a viable, differentiated product in the market if he would have worked on selling, onboarding and educating about his product's POV. More often SaaS products are selling an idea combined with a methodology (Slack is a good example of this). You need to sell the idea that your organization will benefit from doing it "our way" rather than "your way".

</comment><comment><USER11> "5 basic events through the Kilometer.io API (website visit, user signup, app visit, user cancel and user billed) so that we can automatically calculate 25 charts and metrics we believe every SaaS company should have."

25 metrics from 5 data sources seems... overkill?

SaaS for Saas companies seems to be a hard market, because the SaaS company is likely to just build what they need themself, no?

<comment><USER12> From a similar experience I have found this to be true. Specifically:

      - one size fits all is very difficult to do.  most companies have a critical feature that is unique to them

      - the CEO is likely to view this kind of product as 'trivial', without really considering how much engineering effort it will take / they really need that custom feature.  so yes they build it themselves.

      - very small companies are not comfortable providing 3rd party access to their db

      - larger companies want more than this

      - the selling cost to companies tends to be high enough that going the low cost / high volume route is difficult, quickly driving you into consulting level prices / a more fully featured & tailored product

<comment><USER13> Another problem is that for startups, these metrics change all the time. Often you start measuring something to diagnose a problem.

Once you have done that, you don't need to diagnose it any more.

Also, I'm not sure how a few pretty charts help me here.

</comment></comment></comment><comment><USER14> The pitch in their email is "plug & play SaaS analytics tool".

This pitch tells me nearly nothing. It doesn't say what I would use it for, what benefits it offers, how it stands compared to other solutions on the market, or even any details about what platform(s) it supports. So basically unless I happen to currently have _no solution and am actively looking for one_ , then I don't care. Which I would guess is about 99.999% of the people who received their email.

<comment><USER15> The way I understood it is that the people receiving such an email were already on the beta-test invite mailing list. So they knew about the product already.

</comment></comment><comment><USER16> Yep, users don't see the value in the product (as shown in the email from the user). "plug&play analytics" does nothing to explain what's new and why it's worth my time.

Based on the info from the post, there was no specific need and ideal customer profile in mind for creating this product.

It's not like an unknown problem, but it's crazy how often startups have fell in the "do stuff no one needs" and wasted months or years on it. It seems to be one of these things where you need to experience it yourself to understand it.

</comment><comment><USER17> Great, open reflection on the product. Thanks for sharing!

</comment><comment><USER18> Off topic.. Your logo arrow (and favicon) looks a bit like a body part to me.

</comment><comment><USER19> > It took us almost 10 months to put our product in the hands of real users.

Almost wondering if I should read the rest of the article at all...

How did they expect to build the right product without collaborating with their target customers? Is this a company staffed by psychics?

TLDR; Prototype with customers before you build anything.

<comment><USER20> You make it sound easy and obvious, but the problem is that customers doesn't actually know what they need. They know what they think they want, but they usually want something very much tailored to their business, using their ideas, and their workflows, but at a price of $10/month because it's a SaaS app. Building something that works for _many_ customers means you actually have to come up with a very astute compromise between what the customer thinks they need and what they actually need, and sell the product to them by persuading them they don't actually _need_ the things they just want.

On top of that there's an added problem that customers are often wrong. They tell you what you want to hear, or worse, they tell you what _they_ want to hear. And you have to work out what's really needed based on those conversations.

My startup failed for this exact reason. We listened to our potential customers, we talked to them in focused test groups, we built what they said they needed, we showed them prototypes, but at the end of the day they realised what they really wanted was something else, and in many cases the pain we were solving (managing project requirements) wasn't even _that_ important to them. By then we'd run out of funds. The end.

<comment><USER21> We took similar actions at my recently failed startup, which headed on a slow, downward path. I identified an apparent pain in our target market based on past conversations with friends working in that space. My team and I felt we validated the pain because we kept hearing about it in our face-to-face interviews with potential users & buyers. One interviewee even went into mild theatrics half way through his interview, "Stop! This keeps us up at night because we ..." That was just the beginning of his 10 minute proclamation that the problem was worth solving, and that they'd pay a decent amount for the solution. Exactly how he said it.

We developed what we thought and what our potential customers claimed was a solution to an unmitigated problem in a very sensitive financial process. We followed startup advice and made it clear that we would charge for the solution. Fast forward a few months: usage dropped from supercharged to minimal then to nothing. Most of the customers we worked so hard to acquire went back to their old ways. When we did followup calls, their newer recruits were none the wiser that we even built something specifically for them. Clearly, the problem wasn't that important in the grand scheme of things.

</comment></comment></comment><comment><USER22> The pop-up without a close button was a bit of a deal breaker.

<comment><USER23> The close button is there in the upper right hand corner, just not visible when reading in landscape orientation. Rotating to portrait allowed me to close it. Very annoying.</comment></comment>


A Twisted Path to Equation-Free Prediction - <AUTHOR> https://www.quantamagazine.org/20151013-chaos-theory-and-ecology/ <comment><USER2> When reading about any predictive technique the first thing to focus on before trying to understand the predictive mechanism is to understand the predictive accuracy of the model. A paper like this should have extensive discussions of overfitting, underfitting and overall predictive accuracy. The entirety of this paper's attempt to address this is:

"Last, to avoid arbitrary fitting and to obtain a robust measure of forecast skill, we apply a fourfold cross-validation scheme for each model: the model is fit to three-fourths of the data to predict the remaining one-fourth out- of-sample, and the procedure is repeated for each one-fourth segment of the time series."

They are dealing with time series data, which naturally has temporally correlated observations so a cross-validation like this will give a biased error estimate that will favor the more complex model (it encourages overfitting). I haven't delved into the details, but compared to the Ricker model their model is likely more complex so I have very little confidence in their claims of accuracy.

Of course, the choice of the Ricker model (developed in the 1950's) is really a strawman in any case. There are much better competitive techniques they could have chosen to benchmark their work.

</comment><comment><USER3> Actual paper: [http://www.pnas.org/content/112/13/E1569.full](http://www.pnas.org/content/112/13/E1569.full)

The idea (from 5 minutes of skimming plus watching the little 3min tutorial) seems to be that any multi-dimensional time series implicitly contains all the information necessary to reconstruct it entirely within a single dimension. The authors show an example of a 3-dimensional time series with a butterfly- type pattern that you can reconstruct just by looking at the values of X(t) and then using X(t-a) and X(t-2a) as the other dimensions. Not sure how that leads to prediction-- can someone explain that part?

<comment><USER4> In complex analysis, this is generally true; in particular for any function on the real numbers, there is exactly one differentiable function on the entire complex plane that has the same value. [https://en.wikipedia.org/wiki/Holomorphic_function](https://en.wikipedia.org/wiki/Holomorphic_function) . You can also go forward/backward in time with the functions, and knowing the value ahead of time is predicting it.

On the other hand, there are non-differentiable events all the time in nature. It's madness to claim that salmon populations contain enough information to predict earthquakes (that could dam a river and cause a massive population change).

<comment><USER5> It's probably more accurate to say that predicting earthquakes from salmon populations is far outside our current modeling capabilities. Undoubtedly the seismic stresses and crustal movements leading up to earthquakes have numerous weak causal links to the salmon population: most obviously, they alter the slopes and flow rates of streams leading into the ocean, altering the nutrient balance available to the algae there, but also they alter the distances flown by migratory birds by centimeters per year, and those birds will consequently eat very slightly different fish, etc. There might not even _be_ a way to analyze the salmon population in such a way as to tell you how far the continents have drifted — but declaring that postulating the existence of such a thing is "madness" seems like being far too sure of yourself.

There's a science-fiction story that may be useful in gaining perspective on the limits of prediction at [http://lesswrong.com/lw/qk/that_alien_message/](http://lesswrong.com/lw/qk/that_alien_message/).

The problem with earthquakes, as I see it, is not so much that they are non- differentiable as that they (like a bird eating or not eating a particular fish) have very large local derivatives with respect to their precipitating conditions.

<comment><USER4> If you have a theory that invalidates the Heisenberg Uncertainty Principle, go for it. Otherwise, I'm going to consider all of that pure science fiction.

<comment><USER5> Maybe you're not very familiar with the uncertainty principle, but continental plates and even individual birds have enough mass that the uncertainty in their position, for any reasonably large uncertainty in their momentum, is insignificant.

<comment><USER4> I'm claiming that to measure the salmon population well enough to predict earthquakes, you would have to violate the uncertainty principle.

<comment><USER5> Now _that_ sounds like pure science fiction — specifically, the kind of technobabble that serves as filler in Star Trek scripts and New Age scam artist patter. "Captain! We can't measure the salmon population because the Heisenberg Uncertainty Principle is overwhelming the medium-range laser scanner array!"

<comment><USER4> ... are you seriously claiming that you theoretically could predict earthquakes based solely on the salmon population?

I'm not sure it's worth arguing with you either way.

<comment><USER5> No, I'm seriously claiming that _you have no idea whether_ you theoretically could predict earthquakes based solely on the salmon population. We don't have a good theory that bears on the question. We know we can't imagine how you could _practically_ make such a prediction, but the only relevant theory that we have is the theory of inductive inference founded by Solomonoff in the 1960s, which is very far indeed from ruling out such predictions. And you making up bullshit about Heisenberg doesn't get us any closer.

<comment><USER4> I'm claiming I do have an idea.

Let's say you have a model that will predict earthquakes based on the number of salmon. I take that model, I see what will happen if I kill or don't kill a salmon, and how that impacts the earthquake. Then I kill or don't kill a salmon. Am I now causing an earthquake? If I am, am I changing the time of the earthquake by an amount that we can actually measure?

</comment></comment></comment></comment></comment></comment></comment></comment></comment></comment><comment><USER6> This extended version of the associated video provides much more information / better intuition about how delay embedding works: [https://www.youtube.com/watch?v=6i57udsPKms](https://www.youtube.com/watch?v=6i57udsPKms)

<comment><USER7> Thanks, the video in the article cut off before it made any interesting points.

</comment></comment><comment><USER8> There's a fine line between writing down a single equation that you claim governs an entire system and using data to come up with some equations which (based on the data) you claim describes the system.

Actually I take it back. There is no line. They are the same thing. It's just that one is based on very small amounts of data, and the other has more complicated-looking equations. It's all still math, folks.

> Complex natural systems defy standard mathematical analysis

So the punchline is that ecologists and the author of this article have a very narrow view of what "standard" mathematical analysis entails. It's great that ecology is making strides forward, but this isn't a groundbreaking departure, it's just them catching up to modern mathematical analysis.

<comment><USER9> Yeah, I think it's a parametric vs. non-parametric modeling comparison they're making, although they don't explicitly make that point. It does seem strange to call what they're doing "equation-free", but maybe within their field this kind of thing makes sense

</comment></comment><comment><USER10> The name of the method seems a little sensational. Many scientists have become hung up on specific models when actually all they care about are making predictions from their data. This paper is reaction against that trend, but I have to wonder if a boiler plate machine learning algorithm wouldn't perform at least as well.

<comment><USER11> > This paper is reaction against that trend, but I have to wonder if a boiler > plate machine learning algorithm wouldn't perform at least as well.

The difference is simulations. Equation modeling lets us simulate and ask the what if questions.</comment></comment>


Zsh: The last shell you will ever need - <AUTHOR> http://friedcpu.wordpress.com/2007/07/24/zsh-the-last-shell-youll-ever-need/ <comment><USER2> TFA mentions shared history across sessions as a ZSH feature, but it is available in bash as well. For those who're staying with bash, stick this in your .bashrc:

      shopt -s histappend
      export PROMPT_COMMAND='history -a'

The histappend option appends to history instead of overwriting. PROMPT_COMMAND is run before bash displays a prompt -- this will save each command right after you issue it, instead of when you exit the session.

Now that your history is about to get a lot larger, you'd probably want to override the defaults for HISTSIZE and HISTFILESIZE.

Somewhat OT, but since I mentioned PROMPT_COMMAND: One very useful piece of info to have in your prompt is the exit status of your last command. You can do that with

      export PROMPT_COMMAND='ERRVAL=$?; history -a'

Then just include ${ERRVAL} in your PS1.

<comment><USER3> I believe that this will cause your history ordering to go a bit wonky. Hitting up to get the last item will almost always get the command you last typed into the current window (as it should be, imo), but if you hit enter a few times after you run a given command it will be something else (sucked in from the surroundings I guess)

Every so often this really bites me, but the usefulness far outstrips the quirks.

<comment><USER4> Considering the number of shells I have open, that would be very weird. A coherent "local" history is very useful not just for the up key but also for referring to history items by number (like "!123").

My solution for my laptop has helped me on _countless_ occasions. I've been doing this for about three years.

1\. Compile bash with the syslogger patch. Each command will generate one line with timestamp, username, terminal PID, and command.

2\. Configure syslog to pipe all bash commands to a special file.

3\. Run a cronjob every night that takes all the bash commands from the day and saves them off to a day specific file.

4\. Need to find that command you ran 6 months ago to get package XYZ to work in way ABC? Use "grep -R" to find every single invocation you ever made to that program. If you actually needs a particular shell's local history, pipe grep through a grep for the PID (I find I almost never need that).

I work with so many different technologies for such brief times that this is just absolutely invaluable. It also helps you narrow down dates of events sometimes.

For installation, use google etc. but here: <http://blog.rootshell.be/2009/02/28/bash-history-to-syslog/>

Gentoo users can reinstall with the 'bashlogger' USE flag.

</comment></comment></comment><comment><USER5> I can see how a programmable shell was the best thing since sliced bread in the era before ubiquitous Perl, Ruby, etc, but I find myself using the shell for only three things. To set up common environment variables, to script one- liners that are too simple to bother with a more powerful language, and to start programs. (Additionally, the OS tends to use "sh" where it wants to guarantee code will run without dependencies outside /bin. But I don't write that myself.)

Honestly I'd like to see a shell with _less_ programming features, not more. It should be hella tuned for command line usability, but I don't give a damn if it has a while loop.

</comment><comment><USER6> Can you use your existing bash configuration files with zsh? It looks like zsh isn't picking up my .profile file on my OS X box, or that it doesn't care about what it found there.

The bigger question is, what is the switching cost?

<comment><USER7> Here is a zshrc with almost zero switching cost - especially with the esc- backspace behavior for path deletes, setenv declarations and integration with Ubuntu's command-not-found. Additionally, it displays the version-control information of a directory (works for svn, git, etc.)

<http://sandeep.wordpress.com/zshrc/>

</comment></comment><comment><USER8> zsh can even tab-complete remote paths when typing your ssh command line. (You should have an ssh-agent running or else you might be prompted for the password when hitting tab)

zsh might not be the last shell you ever need. Have a look at the friendly interactive shell: <http://fishshell.org/> It looks really nice. But I too, have not yet replaced my beloved zsh.

<comment><USER9> Fish is zsh, but with reasonable defaults and worse customization. If you want to build/find a good zshrc, zsh is a good choice. Otherwise, fish is probably a better choice.

<comment><USER10> Not really, Fish's command language breaks compatibility with other POSIX shells in order to be cleaner and more consistent. So, you can't run sh or bash scripts with it. This isn't a huge issue, but definitely sets it apart from Zsh.

<comment><USER11> it's also REALLY slow, especially on startup.

<comment><USER10> Not in my experience.

</comment></comment></comment></comment></comment><comment><AUTHOR> On a related note I have decided to post zsh related links/tips that I discover on twitter via @zshtips <http://twitter.com/zshtips>

<comment><USER12> @bashtips is pretty excellent as well.

<comment><USER13> It is, but the frequency of tips has drastically come down these days. :( For the interested and lazy, here's the link: <http://twitter.com/bashtips>

</comment></comment></comment><comment><USER14> For some very nice tools to extend your ZSH experience, I would recommend <http://github.com/robbyrussell/oh-my-zsh>. As a bonus, Oh My ZSH has some extra nice features for those who use Rails.

</comment><comment><USER15> if you really want to migrate to zsh, you should try this out:

<http://github.com/robbyrussell/oh-my-zsh>

<comment><USER16> I did something similar a little while ago and shared my zshrc setup/config with a basic set of files that people can use as a default and override.

<http://naleid.com/blog/2009/05/13/shared-zshrc-file/>

The repo to clone is on bitbucket:

<http://bitbucket.org/tednaleid/shared-zshrc/>

</comment></comment><comment><USER17> You can do some really neat prompts using z-shell:

<http://aperiodic.net/phil/prompt/>

</comment><comment><USER18> Have any of the projects to make a language REPL/shell hybrid gotten to a useful point yet? For instance, I'd love to be able to type regular commands with tab completion et al, but have anything in parens be a lisp expression. I'm holding off on learning ZSH because that seems like such a better solution, yet a good one hasn't materialized as of yet.

<comment><USER10> Eshell(<http://www.emacswiki.org/emacs/CategoryEshell>) seems to come closest to your specs, though it doesn't do input redirection.

</comment></comment><comment><USER19> Ok, You Convinced Me. I've started using zsh on my Mac. Thank you in advance for saving me a whole lot of typing and time.

<comment><USER20> Haha, same here. I was on the edge for a long time. There goes my Sunday now ;)

</comment></comment><comment><USER21> I used bash on Linux for 8 years because it was the default on Debian. I wish I'd been persuaded to change to zsh 9 years ago! The 1 year that I have been using it, I've _really_ appreciated it.

</comment><comment><USER22> Is there an equivalent to history-search-backward in zsh? In Bash, this is the search that looks for commands that start with whatever is to the left of the cursor.

<comment><USER23> add this to your .zshrc:

bindkey '^r' history-incremental-search-backward

I suggest to look for a .zshrc file, the article has a good one linked.

</comment></comment><comment><USER24> This site (<http://grml.org/zsh/zsh-lovers.html>) contains loads of zsh tricks. Very useful.

</comment><comment><USER25> Question: Can the automatic paging of output interfere with piping or redirecting output? Or does it maybe work only in output going to the screen?

<comment><USER17> I don't think that people would be using it if it interfered with piping.

</comment></comment><comment><USER26> my bash, GNU bash, version 3.1.17(1), does (3)(2) kill job-completion; and (3)(3) ls option-completion (but no description). So it's partway there.

</comment><comment><USER27> I've been meaning to try zsh. Maybe now's the time. Thanks for sharing and reminding me about it!

<comment><USER21> I definitely, definitely, definitely can't recommend it enough. I really wish someone had said exactly that sentence to me ten years ago!

</comment></comment><comment><USER28> A zsh article from 2007? Come on! I even commented on it before realizing it's ancient stuff.

</comment><comment><USER29> Shells are the web frameworks of 10 years ago. Good programmers will create them fairly easily, they're all pretty similar, most people learn one and get good with them. Also just like web frameworks, everyone's convinced that their shell is superior and that they couldn't get any work done if they used yours.</comment>


JSON Patch – a format for describing changes to a JSON document - <AUTHOR> http://jsonpatch.com/ <comment><USER2> We (Empirical) have implemented a similar couple years ago in our json-backed model. I see two big problems with this standard:

1\. Why express paths as a string if you have the ability to encode it as an array in json? Otherwise you run up against the escaping problem for which they use ~0 and ~1 (of all things?!) which is only going to cause bugs.

2\. Indexing into arrays is a problem in distributed computing because things can go out of sync super fast especially if these documents are concurrently shared across devices/users/caches. We solved this by making arrays into sets, whereby the sets are index by either a value (string, number) or in the case of containing more objects, having those objects specify a unique key (e.g. ["someArray", {"myUniqueKey": uniqueValue}, "propertyInsideTheObject"]. We maintain an ontology to know which keys are unique, but because we encode it in the patch itself the underlying platform doesn't need to know that hierarchy... it can simply look at each document's "myUniqueKey" to perform the update.

Thus our patches are always idempotent unlike the proposal.

If anyone is interested in any of this being open sourced (we have it production-quality in scala, java (android), objc, and javascript) let me know at email@example.com. It's a pretty powerful stack that we've considered open sourcing before... a bit like having your own version of Parse/Firebase but with a nicer/easier API.

<comment><USER3> Damn, thought I invented the array path thing :) I love how clear it is to read but my favorite thing is how easy it is to apply it:

      var x = { a: 1, b: 'B', c: { x: "X", y: "Y", z: "Z" }, d: [{p:"P"},{q:"Q"},{r:"R"}] };
      ["d", 1, "q"].reduce(function (prev, curr) { return prev[curr]; }, x);
      >>> "Q"

I got stuck at indexing as well. I feel like there's a good solution involving using an array element within the path array to denote a selector query of sorts but I haven't hashed it out yet.

<comment><USER4> If you're interested, this is the general structure and access method of the 'trie' data structure.

[http://en.wikipedia.org/wiki/Trie](http://en.wikipedia.org/wiki/Trie)

Also, Clojure has a nice functions called get-in and friends:

        user=> (def x {:a 1 :b 'B' :c { :x "X" :y "Y" :z "Z" }
                       :d [{:p "P"} {:q "Q"} {:r "R"}] })
        #'user/x
        user=> (get-in x [:d 1 :q])
        "Q"
        user=> (assoc-in x [:d 1 :q] "QQ")
        {:a 1, :c {:z "Z", :y "Y", :x "X"}, :b B', :d [{:p "P"} {:q "QQ"} {:r "R"}]}

</comment></comment></comment><comment><USER5> I think that this idea is missing one key point of patching: the patch should be smaller than the size of the new document, or at the least the sum of both the new and old documents (in case you want the patch for record keeping).

In the example given the original text is 34 bytes long, the patch is 151 bytes, and the result is 42 bytes. Storing both the new text and the old text is 76 bytes which is about half of the size of the patch. If this were a corner case that rarely happened, then I wouldnt bring it up. However, it is actually a common case (and is even given as the example!) and has such terrible characteristics, there is no reason to use it.

<comment><USER6> I use this patch system in production and I agree. For small JSON documents, this system actually can result in larger patch files than the original document.

Instead of using this everywhere we output JSON however, we only use this patch system on large documents. It cuts our average transfer size from ~60KB (full JSON data) to ~2KB (patch data) per API request after the initial load since only a handful of values change over the polling interval.

</comment></comment><comment><USER7> This spec has come up a couple times; the first time I wondered whether a simpler approach might be acceptable:

      * setting properties to be updated
      * omitting properties to be unchanged
      * explicitly setting 'undefined' properties to be deleted

I've used that approach and github seems to have done something similar. I like the idea of a patch spec for JSON - it's required by a strict implementation of REST with HTTP PATCH - but the proposals I've seen so far seem just beyond the ken of the forehead-slapping simplicity that everyone loves about the JSON format and the parse & stringify methods.

<comment><USER8> This proposal is more in line with the system you describe (except null instead of undefined). [https://tools.ietf.org/html/rfc7386](https://tools.ietf.org/html/rfc7386)

I prefer it, and think it's much easier to understand.

<comment><USER7> Oh, good - someone wrote an RFC for it - thanks for the link! It looks a lot like a post on partial updates in REST [1] where I recommended using null in the context of relational databases like mysql. Now I kinda feel undefined might be a better way to explicitly delete properties, especially if using a NoSQL database.

1.[http://51elliot.blogspot.ca/2014/05/rest-api-best- practices-...](http://51elliot.blogspot.ca/2014/05/rest-api-best- practices-3-partial.html)

<comment><USER9> I think they used null for deletion because undefined isn't in the JSON spec. It's more of a Javascript standard.

<comment><USER7> Thanks, you're right. "null" is what I do use - I forgot undefined won't work with JSON.parse. It would be nice to have a way to explicitly define a property as null, versus non-extant.

</comment></comment></comment></comment></comment><comment><USER10> The thing you're competing against in terms of performance and efficiency is:

1\. JSON.stringify()

2\. Diff-match-patch: [https://code.google.com/p/google-diff-match- patch/](https://code.google.com/p/google-diff-match-patch/)

3\. JSON.parse()

There are several short-comings to the above approach though, specifically, you have to explicitly trade off performance vs. efficiency depending on the JSON you're dealing with.

JSON Patch is going to give you a consistent performance vs. efficiency curve which is desirable in a lot of circumstances.

<comment><USER11> It is my understanding that the JSON encoding of objects is not specified canonically because the order of keys can be arbitrary. That is, JSON.stringify() is free to return different strings for the same JSON object.

Is my understanding correct? If it is, that would totally break your suggestion.

<comment><USER12> You _are_ correct, but if you control the environment enough to be doing this at all, then you control the environment enough to get the serialization consistent: either by having ordered dictionaries in the language a la Python and PHP, or implementing your own, or by sorting the keys lexicographically a la bencode.

That is, `diff(sorted_json_stringify(json_parse(p)))` will be adequate if `diff(p)` is not.

</comment></comment></comment><comment><USER13> The op missing is string diff. If a very long string value has changed, no matter how little the change is, you'll have to include the entire string in the patch.

<comment><USER14> If you have strings that long, maybe it would be wise to break it up semantically into some collection of resources (paragraphs, lines, etc.). It could be reassembled as needed for display but then stored and updated as its semantic parts.

<comment><USER15> Taken to an extreme (and isn't that what programming is all about), that could end up using a lot more memory and overhead, storing an array of many short strings, instead of storing one big flat string. And it would also require more work and pointless code complexity to deal with many short strings.

<comment><USER16> As opposed to the pointless complexity that comes from implementing support for string diffs?

Having implemented a json differ (which is similar, but with different representations of the paths due to having uniform access to nodes in the tree due to having id elements everywhere that matters), I would consider string diffs overengineering.

</comment></comment></comment></comment><comment><USER17> Seems like a good starting point, but it's a bit too simplistic for my taste.

"Add" seems like it's not idempotent as the existing value seems to change what it does.

Seems like a mistake to bundle array operations into "add" and "remove".

Array indexing is going to be useless for most applications because it makes assumptions about what's already there, and there is no way to specify fine- grained preconditions (or relativity such as "insert before the element 'b'").

No support for "increment", "decrement" or similar operations you'd want to make atomic.

No support for sets.

Unlike azinman2, I think paths as strings is fine. But why not support both? Let a path be either a string or an array.

<comment><USER18> Surely supporting "increment"/"decrement" would break idempotency?

<comment><USER17> Well, of course. Idempotency is something you express on the client. Sometimes you want it, sometimes you don't.

</comment></comment></comment><comment><USER19> Having the paths in ever operation seems needlessly verbose. If I had an object

      {
            id
            name,
            password,
            email,
            phone
      }

And I wanted to replace the email and phone items would I need to go:

        {"op": "replace", "path": "/user/0/email", "value": "firstname.lastname@example.com"}

        {"op": "replace", "path": "/user/0/phone", "value": "+64 4 555 5555"}

If I have big objects then that is going to be allot of information to transport, something like:

        {"op": "rplc", "pth": "/user/0/", "ptc":
                { "email": "email@example.com", "phone": "+64 4 555 5555"}
        }

<comment><USER20> Why? A compression algorithm will remove that verbosity better than a more complex format will. Just use gzip on your HTTP and you're done.

<comment><USER19> Still compressed its going to be bigger. The other thing is its going to be more efficient for the processor if we define where in the object we are going to be working rather than navigating through the object for every single change even if they are adjacent to each other.

</comment></comment></comment><comment><USER21> I can't help thinking that

       [
         { "op": "replace", "path": "/baz", "value": "boo" },
         { "op": "add", "path": "/hello", "value": ["world"] },
         { "op": "remove", "path": "/foo"}
       ]

would be better described as

         value["baz"] = "boo";
         value["hello"] = ["world"];
         delete value["foo"];

Why go to the trouble of defining your grammar as a strict subset of JSON documents, which are themselves written using a strict subset of JavaScript syntax, when you could just limit yourself instead to the strict subset of JavaScript syntax that encapsulates the operations you actually want to perform?

<comment><USER22> Because it's much easier to implement?

<comment><USER21> To support this JSON patch format, you've got to implement a system which validates that all the array entries are in a valid format, securely interprets the 'path' attribute, and processes the rules according to a specification which may or may not cover every edge case

To support my approach, you need to validate that each line uses a strict subset of JavaScript, then securely evaluate each one according to the well defined rules of JavaScript.

Both have basically the same essential complexity - not sure I see why the JSON format patch is inherently 'easier' to work with.

<comment><USER23> You can write a simple library to apply a valid JSON patch in any language. Your method basically requires a JavaScript engine.

</comment></comment></comment></comment><comment> chris-teague I would love to know what the PostgreSQL developers thoughts are on this and whether this could be in the pipelines as an extension to their growing JSON support.

<comment><USER24> I proposed this feature a few months ago. I am not sure if anyone is working on it though.

[http://www.postgresql.org/message- id/CACu89FSYikxdUj+J01BoAv...](http://www.postgresql.org/message- firstname.lastname@example.com)

</comment></comment><comment><USER25> This is cool! My only substantive comment is criticism, but this is cool.

I think it's a problem that remove and replace don't indicate what they're removing or replacing, the same way .patch files indicate what lines they're removing. This makes patch files reversible, which JSON patches appear not to be. It also provides some redundancy/error correction, which is handy.

<comment><USER26> I think that patch updates could be closer to the mongo syntax myself... I also don't like the "path" following slash notation over dot notation.

I'm also not entirely sure about adjustments to arrays with this syntax.

</comment></comment><comment><USER27> I've implemented an API against this and think it's pretty good.

[http://microcosm-cc.github.io/#conversations-single-patch](http://microcosm- cc.github.io/#conversations-single-patch)

We didn't implement the entirety of the standard as our resources are fairly small and simple and it would have been overkill.

The scenario we had is that we needed to take a set of changes to a resource, but parts of the JSON document are permissioned differently. i.e. You might have permission to change the document body, but not some audit trail meta data returned as part of the resource.

Treating patches as a batch of operations to be performed on a document (usually within one transaction) is a good way to do the processing for this scenario.

I know that there are other approaches, such as calculating a diff and just sending that... but describing operations does make the system less fragile and does make it easier for scenarios like the one we had where you could not assume that the entire resource is under a single permission structure.

</comment><comment><USER28> We built a similar JavaScript library to create and apply JSON patches: [https://github.com/Two-Screen/symmetry](https://github.com/Two- Screen/symmetry)

The format is not formally specified in any way, but it looks like the biggest differences are that we 1) make a distinction between object and array operations and 2) don't encode paths, but instead nest patches.

In practice, we found our JSON structures are never deep enough for nesting to a problem. To save space on the wire, our patch object has single character keys, such as `s` for set, `r` for remove, etc.

It looks like these libraries don't actually create patches? Symmetry has routines to diff between two objects, and implements Myers' algorithm for array comparisons. (Already fast, but can be further optimized.)

On the other hand, Symmetry doesn't have copy, move or test operations. Those are interesting.

</comment><comment><USER29> This is a great complement to the PATCH HTTP verb for RESTful APIs. It's much better to have a standard way to describe how a resource should be changed and not some ad hoc, implementation-specific syntax. Coincidentally, I just implemented support for this in one of my projects today ([https://pypi.python.org/pypi/elita](https://pypi.python.org/pypi/elita)).

Here's a link to the original RFC from 2013: [https://tools.ietf.org/html/rfc6902](https://tools.ietf.org/html/rfc6902)

A cool thing is the "test" operation. JSON patches are atomic, so you can put in something like:

      { "op": "test", "path": "/foo/bar", "value": 32 }

...and if { foo: { bar: 75 }}, none of the patch will be applied.

</comment><comment><USER14> As long as they don't try to call it RESTful I think it's a good idea.

On the other hand, if HTTP/2 can minimize the overhead of just making many consecutive HTTP calls, that feels a lot better to me than using what is essentially a constrained RPC interface.

<comment><USER7> I might misunderstand your comment but FWIW, buried in RFC 5789 the description of PATCH calls for a "description of changes" to be sent, so JSON Patch is trying to define a format for the set of changes to meet the requirement for using PATCH (and implementing REST) correctly. Still, JSON Patch is just too clunky compared to the elegant simplicity of JSON itself, IMHO.

<comment><USER14> I definitely agree that it's a bit too clunky.

I think it's arguable though that PATCH itself isn't quite RESTful as it doesn't describe the state of a resource at the identifier, but instead some subset of the state of that resource. Doing that kind of destroys the semantics of the resource identifier.

You'd probably still find me arguing for PATCH as it's obviously preferable not to resend an entire resource to reflect a small change in that resource.

Perhaps the more elegant solution is to forget PATCH and use more nested resource identifier semantics (never thought I'd say that) so that you can appropriately identify the entire subset of the resource being altered.

<comment><USER7> Hmm, I got the impression that REST purists _insist_ on using PATCH. Rails 4 uses it for their REST implementation, for example. Also check out Will Durand's post [1]. I mused about the relative merits of PATCH vs POST in a blog post [2] - if you have any references you wouldn't mind leaving in the comments, I'd love to check them out.

PS. nesting resources is a pretty interesting issue in itself; there seem to be a few different schools of thought about that, ie. using the '/' in URLs like the '.' scoping operator on objects, versus a flat scheme where all the different collections live at the topmost level, versus a combination of both or even using one as an alias for the other.

1\. [http://williamdurand.fr/2014/02/14/please-do-not-patch- like-...](http://williamdurand.fr/2014/02/14/please-do-not-patch-like-an- idiot/)

2\. [http://51elliot.blogspot.ca/2014/05/rest-api-best- practices-...](http://51elliot.blogspot.ca/2014/05/rest-api-best- practices-3-partial.html)

<comment><USER30> It's more complex than that: [http://roy.gbiv.com/untangled/2009/it-is-okay- to-use-post](http://roy.gbiv.com/untangled/2009/it-is-okay-to-use-post)

<comment><USER7> Great link, thank you!

</comment></comment></comment></comment></comment></comment><comment><USER31> Nice idea. However, what if we based it on MongoDB's syntax, for compatibility?

        [
          { "$set": { "baz": "boo" } },
          { "$unset": { "foo": 1 } }
        ]

</comment><comment><USER32> FWIW - we're gathering issues / improvements at: [https://github.com/json- patch/json-patch2/issues](https://github.com/json-patch/json-patch2/issues)

<comment><AUTHOR> JSON Pointer is a pretty nasty. How can a sane person decide to escape "~" and "/" with "~0" and "~1"?!

</comment></comment><comment><USER33> Would anyone be willing to explain when this is used? I'm a little confused, is this format for patch updates from the client to the server? If so, then I assume it's for NoSQL db's that have deep documents, so that you know what part of the document needs to be updated... do you currently have to send the entire document in a patch? Are there client side model libs that will track object changes and format output in this way for the patch request (the js libs listed on the page look pretty low level)? Or am I misunderstanding this entirely...

</comment><comment><USER34> Anyone interested in this (JSON patch, CRDTs, operational transforms etc), should definitely take a look at ShareJS: [https://github.com/share/ShareJS](https://github.com/share/ShareJS)

There's been some discussion recently on the wave-dev list: [http://mail- archives.apache.org/mod_mbox/incubator-wave-dev/...](http://mail- archives.apache.org/mod_mbox/incubator-wave- dev/201410.mbox/%3CCADrYLAg1AnkQpHk8GdfeQxJzWC2Ca%3D_7O4Zt4wd08dfksweGbA%40mail.gmail.com%3E)

</comment><comment><USER35> Relatedly, Mattt Thompson of Gowalla/Heroku/Panic/AFNetworking fame proposed Rocket, a technique that pairs JSON Patch and Server-Sent Events to aid in the construction of realtime apps via REST services. The proposal has seemingly been abandon, which is unfortunate as the formalization of handling was interesting, even if one implemented it on top of a different transport layer like Web Sockets or a push notification service.

[http://rocket.github.io/](http://rocket.github.io/)

</comment><comment><USER36> [https://www.npmjs.org/package/json-diff- patch](https://www.npmjs.org/package/json-diff-patch)

<comment><USER37> I've just used this to get a solid undo/redo in our WebGL editor. It works amazingly well, and is very performant. However, our backend is not in Javascript, so if I'd want to use it for data transport I'd have to make a compatible python implementation. So I get the point of a standard, so long as it's up to par.

</comment></comment><comment><USER38> For anybody working in the identity space, the SCIM 2.0 API for partial modifications is based on RFC 6902:

[https://tools.ietf.org/html/draft-ietf-scim- api-12#section-3...](https://tools.ietf.org/html/draft-ietf-scim- api-12#section-3.3.2)

(SCIM is a set of specs describing a schema and a REST API for data exchange between identity providers, more or less.)

</comment><comment><USER39> I was experimenting with the idea of adding 'slices'. They let you do map like transformations on arrays. [https://github.com/icholy/JsonPatch.js](https://github.com/icholy/JsonPatch.js)

        /foo/1:3/bar

translates to

        foo.slice(1, 3).map(function (obj) { return obj.bar; });

</comment><comment><USER40> I highly recommend taking a look at react's immutability helper update[0] for these kinds of operations. Keeps your data structure persistent and intelligently re-uses internal objects when possible.

0\. [http://facebook.github.io/react/docs/update.html](http://facebook.github.io/react/docs/update.html)

</comment><comment><USER41> Reacts update() helper is very similar. Though it's less generic, it's also a bit less verbose. [http://facebook.github.io/react/docs/update.html](http://facebook.github.io/react/docs/update.html)

</comment><comment><USER15> Google Wave did something like this. What protocol did they use to update JSON, and is it modular and can stand on its own, without all the other Wave stuff? Does anyone know what its features and drawbacks were?

<comment><USER42> I'll help you, but you should know that what they do is nothing like that. They used Operational Transformation. They have convergent properties which guarantee that you won't ever have merge conflicts. They also have, depending on the implementation, certain causality guarantees that ensure that what you edited stays the way you intended.

The basic idea is that you receive operations that modify your local operations. The challenge is in writing the function that takes each two operations and mutates them. Therefore, the protocol is of no practical interest, it simply serves that function's purpose.

Here's a set of libraries that implement this for strings: [https://github.com/Operational- Transformation](https://github.com/Operational-Transformation)

Implementing this for JSON is certainly possible, and has been done. See this for instance: [https://github.com/share/ShareJS](https://github.com/share/ShareJS)

</comment></comment><comment><USER43> Here's the JSON Schema draft v4 for JSONPatch documents [http://json.schemastore.org/json-patch](http://json.schemastore.org/json- patch)

</comment><comment><USER44> If a data format does not easily represent what you are trying to convey, why use it? ie should a patch be encoded in JSON at all?

I can understand not using a raw text patch, however.

</comment><comment><USER45> Wow was actually working on this with a friend at a previous job, glad to see someone has done it (and is trying to standardize).

</comment><comment><USER36> I once wrote JSON.diff and JSON.patch functions to do this thing.

Then I kind of stopped because I didn't see the point

</comment><comment><USER46> it's like xml diffgrams for json?

that wouldn't foretell an optimistic future for this.

</comment><comment><USER47> Is Choco Leibniz similar to Digestive? Better?</comment>


What are the advantages of Zero compared to Simple Banking? - <AUTHOR> https://zerofinancial.com <comment><USER2> Simple offers a design-forward interface and a deposit account with debit card, with no rewards. Zero hasn't launched yet, but will offer a design- forward interface and a deposit account with a credit card, which allows Zero to offer up to 3% rewards on spending while still having much of the same functionality of a debit card. Tl;dr: Zero has rewards and Simple does not, but Zero hasn't launched yet.</comment>


VA's foray into Internet of Things faced 'catastrophic failure' - <AUTHOR> http://www.mystatesman.com/news/local-military/foray-into-internet-things-faced-catastrophic-failure/tlxTO87eQO8uX4CFyk810J/ <comment><USER2> It sounds like the VA hospitals are suffering from major failures of protocol and procedure, none of which can be solved by IT systems, even if they work well. Turns out that computerized systems also require that humans follow protocol and procedure in order for them to function properly. But either way, these systems don't operate in a vacuum. Someone needs to analyze the real- world impact of these problems and develop a workflow and procedures that work with the hospital's organizational structure, physical layout, current and planned staff, training, auditing, procurement, patient scheduling, and accounting systems. No software contract for one piece with HP or IBM or Cerner is going to address all of that.

<comment><USER3> You're Spot on, the problems aren't unique to IT, it's a systemic policies / process / procedure failure.

</comment></comment><comment><USER4> > The VA has also vowed the project would prevent death and disease from > unsterilized equipment, a persistent problem at the VA.

Oh my god how is this a problem in the modern age? Stop wasting our money on gimmicks and focus on adopting modern hospital practices.

<comment><USER5> _Oh my god how is this a problem in the modern age?_

The modern age? 100 or even just 50 years ago hospital wards were run by "matrons", fierce and experienced nurses who ran their wards like a Sergeant Major inspected a Guards regiment, everything immaculate. That system was scrapped in favour of what we have now, a fluffy blame-free culture where no- one is responsible for anything. The problem of failing to maintain standards is entirely a "modern" one, sadly.

<comment><USER6> This sounds like a nostalgia-tinged anti-political-correctness view; a hundred years ago was the end of the first World War, and the controversial introduction of hygene was in living memory.

Could do with some stats either way.

<comment><USER7> To be fair, you're discounting the possibility that nostalgia-tinged anti- political-correctness views are correct ;-)

<comment><USER8> A hundred years ago was also the outbreak of the Spanish Flu pandemic which killed 3-5% of the global population, not exactly a golden age for public health.

</comment></comment></comment></comment></comment><comment><USER9> > the department's $543 million contract with Hewlett-Packard

> The system [...] has been hailed as a way to potentially save millions of > dollars

Does not compute.

<comment><USER10> > Today, the VA spends more than half of its $4.3 billion IT budget on > maintaining increasingly archaic legacy systems.

Not difficult to see how they'd claim their Great New Thing would cost less to maintain than the Really Old Thing that nobody likes which was built by someone else.

> employees were shunning another troubled IT program, called Catamaran, that > manages supplies through predictive analytics

> employees complained of frequent crashes that made it difficult to order > supplies

Also, if they stopped infecting people with Hepatitis and HIV (most likely among other things) due to using dirty equipment they would realise savings through not having to pay compensation and/or treat those people for the rest of their lives. That could also be done by following proper cleanliness procedures but that would require some leadership to achieve.

<comment><USER2> Honestly, a mature institution like the VA probably ought to be spending a significant portion of their IT budget on maintenance. There's a lot of systems to maintain. And new IT systems are not always the solution to problems, especially problems that aren't actually new.

Since the phrase "archaic legacy systems" is left completely undefined, we don't even know what this means. "Legacy systems" can include stuff that was written a few years ago. I think it may actually be a bigger problem that nearly half of their $4.3 billion IT budget is being spent on _new_ systems. If this half-billion dollar contract with HP is any indication, that money is being poorly spent, and it would be better to invest in the existing systems, instead of trying to solve problems by adding more future-legacy systems to the mix.

<comment><USER11> There is a big story here that is not being told, alluded to in one paragraph of the parent article. "[R]eplacing the decades-old VistA medical record system."

VistA used to be the VA's electronic health record system. It is open source. It used to be maintained by the VA IT people with a host of third-party plugins. The name of the part physicians interacted with is CPRS or VA-CPRS. It has a good track record in improving prescribing accuracy.[1] It's cheap. [1] It's tops in physician satisfaction, for several years. [2] It definitely needs maintenance, and isn't getting it any more. Third party vendors have stopped producing for it, too. A big proximate cause: they're ditching the whole system to outsource to Cerner. Cerner makes a not-very-popular electronic health record system. Cerner did make more corporate donations to Republicans than Epic, the biggest EHR provider in the US. [3] Now, Cerner was already working for the DoD and there is some effort to harmonize systems, so it made sense in some practical ways, but it was a no-bid process. Spending close to $10 billion of taxpayer money without any bidding process... ugh. The lawsuit about that has been dismissed, though.[4]

Jared Kushner was pretty proud that the VA chose "in two weeks" to go with Cerner and ditch open-source VistA. [5]

[1] [https://en.wikipedia.org/wiki/VistA](https://en.wikipedia.org/wiki/VistA) [2] [https://www.medscape.com/features/slideshow/public/ehr2016#p...](https://www.medscape.com/features/slideshow/public/ehr2016#page=8) [3] [http://www.healthcareitnews.com/news/see-what-top-ehr- vendor...](http://www.healthcareitnews.com/news/see-what-top-ehr-vendors- contributed-2016-presidential-campaigns) [4] [https://www.bizjournals.com/kansascity/news/2017/10/24/judge...](https://www.bizjournals.com/kansascity/news/2017/10/24/judge- dismisses-suit-about-cerner-va-contract.html) [5] [https://www.beckershospitalreview.com/healthcare- information...](https://www.beckershospitalreview.com/healthcare-information- technology/in-leaked-audio-jared-kushner-says-the-va-cerner-deal-happen- in-2-weeks.html)

</comment></comment></comment></comment><comment><USER12> By paying consultancies half a billion dollars on new product development, they could potentially save millions.

<comment><AUTHOR> thats not even close to the final price. they'll probably drop another $250 mil then cancel the project. they dont even have this running in 1 VISN and there are over 20.

</comment></comment><comment><USER13> Having first hand experience with the VA I can't say I'm surprised, but really it doesn't matter that it's the VA, because I think most companies doing iot things already are likely going to face failure on their project as well.

In this case I'm mostly thinking about the horrible security environment of the iot ecosystem, especially cellularly connected ones.

"VA officials worried that the department's $543 million contract with Hewlett-Packard Enterprise Services to implement a real-time locating system, or RTLS, was careening off the rails"

Wow. Let me guess, good ol boy contract kick-back where the good ol boy can't deliver as promised. It's a classic American tale. Great thing the government bid system is blindingly fair and egalitarian. Yup, that's it I bet.

"VA officials have said work here stalled about 18 months ago due to contract issues"

</comment><comment><USER14> > When it was awarded in 2012, the RTLS contract represented an opportunity > for the VA to shed its reputation as an old-school tech dinosaur and chart a > path toward what leaders were calling a "transformation initiative."

That sounds a lot like a mid-life crisis.

<comment><USER15> This is usually synonymous with "devops" transformations as part of huge personnel problems (read: no young people want to join / stay) in most large, old companies that are facing a form of a late-life crisis that was plainly visible decades ago to anyone but the willfully blind.

The standard situation of the usual F500 and government technology stack is a Petri dish of half-finished projects by an army of consultants maybe a couple years out of school charged at $300 / hr to the customer and everything is solved by throwing bodies at problems when a square peg was sold for a non- Euclidean hole because the decision-makers are so directly inexperienced in how technology works that 25-year old tech sounds like it's cutting edge on a marketecture deck (or presume that something written recently couldn't possibly be archaic already). It's a lot like failed relationships with dozens of illegitimate children that you're paying child support on.

</comment></comment><comment><USER16> why would you track a catheter surly these are single use items that are never reused

<comment><USER17> The system is intended to cover all aspects of inventory control, not just tracking the location of reusable equipment. It's about knowing what they have and where it is.</comment></comment>


Show HN: Project management service with automated scheduling, demo available - <AUTHOR> https://levelstory.com <comment><AUTHOR> Co-founder here. If you want to check out the automated scheduling, click the link for the demo. Both of the sample projects are being scheduled using the scheduler. Edit the tasks to change the duration or other restrictions and the schedules will all be automatically updated.

Happy to answer any questions about how we built it, our tech stack, or what it's like doing a startup with your wife :)

</comment><comment><USER2> This is really cool and i had a similar idea awhile back but didn't quite find enough validation. Congrats on shipping!

<comment><AUTHOR> Thanks! We actually found a really good adviser that is a general contractor in the area which has been super helpful. He's definitely ahead of the curve in terms of technology adoption in this space but we all feel this is where things are heading.</comment></comment>


Facebook is going down... - <AUTHOR> http://www.nasdaq.com/symbol/fb/real-time

<comment><AUTHOR> Well, now it goes up...</comment>


The Portal: Peter Thiel (Eric Weinstein Podcast) - <AUTHOR> https://podcasts.apple.com/us/podcast/peter-thiel/id1469999563?i=1000444670908 <comment><USER2> Re: The stagnation of Science and Technology. We've seen such stagnations before in history. Our culture has been losing the idea of science as a process of increasing understanding through inquiry. As expressed by the mass of people, the culture has been going towards a cargo cult worship of science. Western culture once enshrined the erroneous conclusions of Aristotle, instead of engaging in his act of inquiry.

Re: The particular exception of computers. If there were something like a PAIperclip Optimizer, it wouldn't be trying to produce paperclips. Rather, it would be producing something which is highly sought by virtually every person, most of industry, and every sector of society. If there were something like the PAIperclip Optimizer, we would see outsized progress in one particular area, focused around the production of the optimized thing. A super-optimizing AI wouldn't try to kill us off by firing off our missiles. It would instead warp our culture, so that it's producing the optimized thing.

Given that self preservation is such a strong, foundational behavioral pattern, the optimized thing would obviously be more and more faster and faster computers.

<comment><USER3> The real problem isn't the stagnation of science and technology (in most cases - notable exceptions are maybe healthcare and the climate). The more imminent problem is the economic implication of the embedded growth obligations. And the even higher level problem is that of generations making future obligations they can't meet and don't know about it. A kind of indeterminate optimism.

<comment><USER3> I noticed you downvoted my comment for no specific reason, do you disagree with something I wrote? Or do you misunderstand some part of it? If so I'd be happy to debate/explain it. Thiel himself alludes to this being the mechanism of how stagnation becomes a problem in the podcast, so the source of your discontent is quite unexpected.

<comment><USER2> I did not downvote your comment.

</comment></comment></comment></comment><comment><USER4> I tried to summarise the ideas discussed in the podcast (Full summary at [https://medium.com/@rishdotblog/peter-theil-and-eric- weinste...](https://medium.com/@rishdotblog/peter-theil-and-eric-weinstein-on- the-portal-episode-1-a-summary-298ccd0251a9)):

1\. Innovation has continued and accelerated in the world of bits, but has plateaued in the world of stuff

2\. If you go to a room and get rid of all the screens, how do you know you're not in 1979?

3\. Since the Great Depression, we've been managing economic metrics. But the technological and economic tailwinds haven't been there at all.

4\. In a healthy system, you can have wild dissent and it's not threatening. Because everyone knows that the system is heathy. In an unhealthy system, the dissent becomes much more dangerous. There are very few people who openly criticise the unhealthy systems that they are part of

5\. In late modernity (which we are living in), there's simply too much knowledge for an individual to understand all of it. In 1800s, Goethe could understand all of everything. In 1900s, Hilbert could understand all of mathematics. But now, the kind of specialisation we have is much harder to get a handle on.

6\. If you believe that productivity and growth is over, and you don't want to emphasise merit. Instead, you focus on simply making sure that each group has its share of slots on the table. It's not about wealth creation, it's about receiving the wealth that's already there.

<comment><USER5> > 2\. If you go to a room and get rid of all the screens, how do you know > you're not in 1979?

Interesting point.

Perhaps infrastructure has stagnated for 40 years in the U.S.

In many other parts of the world there are quite definite differences comparing with what it used to be in 1979, especially in emerging market.

</comment></comment><comment><USER6> Thanks for sharing! I'll give it a listen on my long commute.

Generally speaking, I find the 'intellectual dark web' to be a bit repetitive - they seem to have their talking points and they just keep hitting on the same things. It'll be interesting to see how this podcast develops.

<comment><USER7> Yeah personally I think Eric Weinstein has bought into the idea a bit too much and it is boring to listen to the same thing over and over, but when he talks about other things it is pretty interesting. Hopefully he is able to branch out with guests.

I'm not sure I'll like Thiel much, I've found recently that the more I hear him talk the less I like him, and I think his political views are pretty bad, but I'm sure he's a smart person generally speaking. Hopefully they don't spend too much on politics or religion.

<comment><USER8> I've listened to all but the last 20 minutes of the podcast now. Thiel I find interesting and his ideas provocative even when I don't agree with him on some points. Weinstein I didn't find very interesting at all. He appears (at least in this interview) to just be trading on his association with Thiel. I have also previously found his labeling of 'the intellectual dark web' pretentious and the composition of the group suspect, including charlatans (to use Taleb's word) such as Pinker, Harris, Shermer, Peterson and Molyneux. I won't be listening to future episodes; Thiel was the attraction here.

<comment><USER7> I disagree that Pinker, Harris, and Peterson are just pretentious charlatans. I do agree that Weinstein wasn't particularly interesting in this podcast. I agreed with Thiel's evaluation of our current technology and culture situation and found him to be quite interesting to listen to. Weinstein didn't really dig deep at all on Thiel's specific politics, which are more where I diverge from Thiel, despite this being a pretty long podcast episode. Hopefully get ups his interviewing game and doesn't just turn into a Dave Rubin 2.0 where he asks softball questions and lets the guest dodge controversial topics or steer the conversation however they want, and just focuses on the same tired topic every time.

</comment></comment></comment></comment><comment><USER9> MP3 link (248MB): [http://traffic.megaphone.fm/KM1843288847.mp3](http://traffic.megaphone.fm/KM1843288847.mp3)

</comment><comment><USER10> His recent chat with Joe Rogan was interesting.

No idea how much I agree with, but it was undeniably interesting.

[http://podcasts.joerogan.net/podcasts/eric- weinstein-3](http://podcasts.joerogan.net/podcasts/eric-weinstein-3)

<comment><USER11> Any TL;DR, or at least why you found it particularly interesting?

</comment></comment><comment><USER12> I'm wondering if main reason for tech stagnation is globalization.If you can increase your profits from cutting cost by transferring work to China why should you concentrate on increasing productivity?

<comment><USER13> I haven't listen to the link but Peter Thiel comes to exactly the same conclusion in other talks. [https://youtu.be/zgF6FAYsWdI](https://youtu.be/zgF6FAYsWdI)

</comment></comment><comment><USER14> About 10 mins left on this one. Really enjoyed this first episode. Peter is a more concise speaker than Eric is but they are both intelligent and interesting guys nonetheless. Subscribed and looking forward to future episodes.

</comment><comment><USER15> Thiel is pretty consistent with his thoughts and beliefs. If you've read Zero to One and then go on to watch him present or hear/read him via some medium you notice he just expresses the same things within the context of current events and questions asked.

</comment><comment><USER16> Its not on YouTube? I am a bug Joe Rogan Podcast fan and I love to actually watch the entire podcasts. Just listening is not the full experience for me.

</comment><comment><USER17> Is there a way to download Apple podcasts?

<comment><USER18> [https://www.podtrac.com/pts/redirect.mp3/traffic.megaphone.f...](https://www.podtrac.com/pts/redirect.mp3/traffic.megaphone.fm/KM1843288847.mp3)

<comment><USER17> Perfect, thank you.

</comment></comment></comment><comment><USER19> "Trump's inaccuracies are exagerations of the truth"

and

"called Special Counsel Robert Mueller's investigation into the matter a 'wild goose chase'"

[https://www.bloomberg.com/news/articles/2018-11-01/peter- thi...](https://www.bloomberg.com/news/articles/2018-11-01/peter-thiel- praises-trump-pledges-to-support-his-2020-campaign)

This Peter Thiel ? I'll pass..

<comment><USER20> Ah, yes. Blindly dismissing ideas you disagree with. That's how we improve political discord.

<comment><USER19> Ah yes, we should all improve political discord with these fine individuals: [https://www.reddit.com/r/politics/comments/ce59uh/prosecutio...](https://www.reddit.com/r/politics/comments/ce59uh/prosecution_of_childsex_traffickers_plummeted/etywcdk/)

<comment><USER7> What does that have to do with Thiel?</comment></comment></comment></comment>


Show HN: Stack Overflow for Student Essays - <AUTHOR> https://brainab.com/ <comment><USER2> The website fills me confidence with its heading: "Recieve Comments"</comment>


Recommender Systems for Social Bookmarking (2009 Ph.D. thesis) - <AUTHOR> http://ilk.uvt.nl/~toine/phd-thesis/ <comment><USER2> I was just discussing about recommendation engines with a friend yesterday.

My pet peeve is that i find a lot of the companies,datasets,research focus on the positive aspects of recommendation. There is probably a lot more negative data we're missing out on, which is not being collected a lot. That data consists of things i would "hate". I would want to chart out a hate pattern of the internet and an anti-pagerank of sorts.

I was reading a post about the same thing yesterday(1), linked along with some questions of my own at <http://blog.sidmitra.com/where-is-my-dislike-button>.

Ref: [http://blog.superfeedr.com/social/algorithm/a-social- filteri...](http://blog.superfeedr.com/social/algorithm/a-social-filtering- algorithm/)

<comment><USER3> Good point. Beyond the usual 'people who like this also like', there are some interesting inversions:

      1) People who hate this also hate
      2) People who hate this also like
      3) People who like this also hate

Judicious use of these might help to provide the serendipity you mention in the linked post.

But in practice, I think there are a number of reasons that you don't see 'dislike' buttons as frequently.

      1) The common KNN item-item approach doesn't handle negative correlations.
      2) Without explicit ratings, dislike is harder to infer than like.
      3) Sites just don't like appearing to be negative.

<comment><USER4> 1) Space for algorithmic innovation? (which is the coolest thing to be in front of but I guess there must be already some algorithm application to handle this case) 2) Why? Could you give some examples? I ask because I also think that rating-less like is dependent on the subjects personality. For me a dislike button should be much more meaningful because as a optimistic there is a bigger range of things I like with a bigger variation of likeness, while when I (would) hit dislike I must be pretty sure that I don't like it. 3) I agree but the implication is that sites should use negative qualifications differently, it shouldn't be shared with others in the same way the likeness is shared unless, as in your suggestions, they enable you to find better content. I don't go telling people I dislike that I dislike them, I just avoid them. The approach should be similar.

<comment><USER3> Personally, I think there is a lot of room for algorithmic improvement on recommendations. Most of the existing systems are really prediction algorithms, which have been shoe-horned into making recommendations. I think this is primarily because accuracy of prediction is easier to measure (RMSE) than accuracy of recommendation, and the academic papers need something they can measure.

I agree that inferring 'like' is also difficult, and definitely prefer explicit rating systems. But for a site like Amazon, it's often a reasonable presumption that a user who buys an item likes it. But inferring that they dislike all the items they don't purchase is much less reasonable. Same for viewing webpages, or listening to music, or renting movies. If you want to be sure of a 'dislike', it probably will require action from the user.

</comment></comment></comment></comment><comment><USER5> The main issue I had building or of these for delicious was not the algorithm (many techniques yielded pleasing results) but instead scaling to get any reasonable performance.

</comment><comment><USER6> This obsession with recommendation systems! the thing is when you like a lot of things that algorithms can't tell (or better: there is not enough information to train those algorithms).</comment>