Functional Programming Doesn't Work (and what to do about it) (2009) - <AUTHOR> http://prog21.dadgum.com/54.html

<comment><USER2>
      Imagine you've implemented a large program in a purely functional way. All the
      data is properly threaded in and out of functions, and there are no truly
      destructive updates to speak of. Now pick the two lowest-level and most
      isolated functions in the entire codebase. They're used all over the place,
      but are never called from the same modules. Now make these dependent on each
      other: function A behaves differently depending on the number of times
      function B has been called and vice-versa.

That sounds like a terrible design decision.

Having two functions which are used all over the place by different modules, and then linking their behaviour to some piece of state that wasn't passed into either of them is a terrible idea. Suddenly that state becomes an implicit argument AND return value, and I have to know about that before I can call the function… Not only that, now I've lost my ability to reason about the behaviour of the function independently from the execution of the program. Further, functions A and B are now impossible to test in isolation. I need to actually call A multiple times before I can test B properly. Then once I've done that, well I've already called B a few times, so how do I test A? I'd have to have assertions for the behaviour of A and B tangled up in the same tests…

This is a mess, whatever programming language/paradigm you prefer.

<comment><USER3> No, the example is unmotivated, but not unreasonable.

For example, say the two functions are computationally expensive, and you want to make them faster. The "piece of state" is a shared cache, i.e. memoization. Furthermore, there's reason to believe that if function A is called with value X, then function B will not be called with Y anytime soon. So if A(X) is called, Y(B) ought to be evicted from the cache.

This is awkward to implement in pure functional languages, but it's not a terrible idea. You can still reason about the functions, test them in isolation, etc.

<comment><USER4> Your example is very different. It seems you're still describing a pure API: testing those new function implementations can be done with the same test suite used for the slower variants.

In the blog post the author choose to specify that the observable behaviour is not pure.

<comment><USER5> What if the cache involves IO? The behaviour is then assuredly not pure.

<comment><USER6> What sort of a cache would use IO? Not any cache I've heard of. (Genuinely interested)

<comment><USER6> I guess I should clarify and say "What sort of a cache would use IO and make a pure function impure?". Even if the cache is so large that it doesn't fit in memory, its behaviour would still be referentially transparent.

<comment><USER5> That's only if you don't count the IO itself as a side-effect.

Does a language like Haskell give you the choice? If so,would it be idiomatic?

</comment></comment></comment></comment></comment></comment></comment><comment><USER7> "Imagine you've implemented world peace. Now let's try to drop couple of nukes in the middle of this..."

Anyway, the point of functional programming is to have as many "immovable parts" as possible, because mutation stands in for parts in motion. And everyone in engineering knows parts that move are usually the first point of failure.

You might need a moving part here or there in a usable design eventually (not always) but you should do you darn best to have them isolated and as few as possible.

<comment><USER8> Aha! Moving parts are failure-prone because of friction. And friction happens when something's changing under you. Which means mutation. Nice analogy ;-)

</comment></comment><comment><USER9>
      Imagine you've implemented a large program in a purely functional way.
      All the data is properly threaded in and out of functions, and there are no truly
      destructive updates to speak of. Now pick the two lowest-level and most isolated
      functions in the entire codebase. They're used all over the place, but are never
      called from the same modules. Now make these dependent on each other: function A
      behaves differently depending on the number of times function B has been called
      and vice-versa.

      In C, this is easy! It can be done quickly and cleanly by adding some global
      variables. In purely functional code, this is somewhere between a major
      rearchitecting of the data flow and hopeless.*

That's the actual problem with C -- people can (and therefore they do) make remote parts of the program depend on each other. One global is fine, but then comes another one, ... if there are two very remote, hidden functions that suddenly need to depend on each other then yes, it's OK if this is hard because you are fundamentally changing information flow in your system.

<comment><USER2>
      …you are fundamentally changing information flow in your system.

And this is the key point. That the language/paradigm adds friction here is _a good thing_ ; it guides you towards a more appropriate implementation.

</comment></comment><comment><USER10> > In C, this is easy! It can be done quickly and cleanly by adding some global > variables. In purely functional code, this is somewhere between a major > rearchitecting of the data flow and hopeless.

"Cleanly" and "global variables" never mesh.

> Here's a simple statement:

        if (a > 0) {
            a++;
        }

> In single-assignment form a new variable is introduced to avoid modifying an > existing variable, and the result is rather Erlangy:

        if (a > 0) {
           a1 = a + 1;
        } else {
           a1 = a;
        }

Er. Plenty of functional languages will let you shadow existing bindings:

        let a = if a > 0 then a+1 else a

However, if you held a reference to 'a' before the shadowing, its value will not change.

</comment><comment><USER11> Functional programming has some serious problems, but that it doesn't allow quickly changing behaviour by introducing global state isn't one of them (I'd say its one of the features).

It takes a whole lot less mental effort to iteratively produce a bad solution in OO, than a good solution in FP.

I don't have anything to back this claim up, but to me it seems the "solution space" in FP is a lot smaller than in OO. As soon as you have mutability everywhere, you are just one more if() or more one state away from achieving what you want. In OO there are a million bad solutions to every problem, so the chance that a group of mediocre developers eventually stumble over one of the OO solutions to a problem is pretty good. It's so good that you can almost be sure that given enough time, any group of developers will reach the goal of a working program (for some definition of working). This makes it appealing to business.

<comment><USER12> Having recently completed my first (small) Clojure project for a client, I don't think the solution space is small unless you're talking about a pretty constrained problem. Given a broad enough scope, there are many approaches/solutions in FP that can yield similar results.

</comment></comment><comment><USER13> This sort of thing stems from fundamentally different ways of thinking about the task of programming, depending on what kind of problems you're tying to solve. Let me sketch some categories of programmer:

Mathematician: primarily concerned with numbers and the stateless universe of mathematical objects. Primary audience for functional programming.

Roboticist: because "control systems engineer" doesn't trip off the tongue. Primarily concerned with physical machinery. Trained on things like Kalman filters and lift controller state machines. Would like the certainty of functional programming but tends to write sphagetti assembler instead in order to reduce interrupt latency.

Plumber: someone who would like to build a system by joining together parts. Perpetually disappointed by how hard this is. Labview is the archetype system here, although a lot of web framework/"stack" talk shows this kind of thinking.

Bureaucrat: someone for whom the records are the primary concern. Tends to build systems which look like CRUD but actually embed a complex workflow which is divided between humans and computers.

The thing is, only the mathematicians are really comfortable hiding all the state in the world behind monads. "Plumbers" might be the next easily persuadable, as FRP maps quite well to this way of thinking, as does map/reduce. But there isn't quite the tooling to enable building _software_ out of pieces joined by pipes.

In the electronic and bureaucratic control systems, _changing state_ is the point of the program. The thinking revolves around state, when it should be updated, and what it should be updated to. Stuffing this all behind a monad results in unnatural-feeling programs. There's a hybrid point where you have a "state" part of the program and a pile of "next state" logic which is basically the only way to write Verilog/VHDL - and it's hard to learn and not so naturally expressive.

Perhaps the way in is something like C++'s "const": you take a language that isn't functional and mark parts of your code as non-state-modifying, making it easier to subject them to formal reasoning.

<comment><USER14> I appreciate this kind of distinction. However, I wouldn't be so quick to say that purely functional state transitions are unnatural or difficult. If you just formulate it slightly differently, it makes lots of sense: the point of the program is to calculate the next state given a current state and some input. And that's natural to formulate as a pure function of type (Input, State) → State.

Let's also not overestimate the elegance, naturalness, or correctness of the typical imperative approaches to control systems... Most programs are already difficult to understand, think about, predict, and change. And a huge reason is that they tend to lack principles. How often do you look at code and feel like there's no reason to believe that it works correctly, other than the fact that it seems to? And then you look at the growing bug count in the issue tracker...

State monads are (from one perspective) an implementation pattern used in Haskell to encapsulate state transitions. They're not necessary, and actually they're mostly used because they enable some syntactic sugar.

I've written very stateful code in Haskell, professionally, and it's been quite nice. No worse than any other language, and nicer in many ways. STM and light-weight threads go a long way.

I wish I had an example of a well-designed CRUD-ish system in Haskell to point to. Maybe there are some. I haven't had the opportunity to make one, but I feel confident that some very nice design patterns could emerge.

To give some indication that semi-nice stateful things can be written in Haskell in natural styles, here is a very simple component of an "IRC relay" system I never quite finished. [1] It's an independent process that reads IRC messages from a RabbitMQ queue, and writes "PONG" for every "PING". It's based around the `pipes` library and as you can see the basic effect of the program is explicitly stated in the main function.

[1]: [https://github.com/mbrock/klatch/blob/master/Klatch/Ponger/P...](https://github.com/mbrock/klatch/blob/master/Klatch/Ponger/Ponger.hs)

</comment></comment><comment><USER15> It seems to me there are several issues:

1 - You don't know what's happening. Even with higher level imperative languages you have a (direct) correspondance between code and machine code (or something similar like bytecode). Not in functional

2 - Sometimes you want side-effects, in ways you didn't know they were side effects (like writing/reading a file, random numbers, etc). Now Haskell's way of solving this may be mathematically elegant, but it's clunky in real life.

3 - Doing things functionally require a different mindset. You think of a for/if cond() break, and you need to rethink this on functional (and a lot of other things).

<comment><USER16> > 1 - You don't know what's happening. Even with higher level imperative > languages you have a (direct) correspondance between code and machine code > (or something similar like bytecode). Not in functional

There is a direct correspondance between Haskell code and machine code. GHC translates from Haskell into an imperative language called C--, which is basically a restricted subset of C. Sure, the translation is of a different style than you are used to in the imperative world, but it's still a direct translation.

> 2 - Sometimes you want side-effects, in ways you didn't know they were side > effects (like writing/reading a file, random numbers, etc). Now Haskell's > way of solving this may be mathematically elegant, but it's clunky in real > life.

Arguably, but after several years with Haskell I now find unrestricted side- effects "clunky" and effect typing much more practical.

> 3 - Doing things functionally require a different mindset. You think of a > for/if cond() break, and you need to rethink this on functional (and a lot > of other things).

I agree. It requires a significant change to your way of thinking about programming.

<comment><USER8> Well, all languages translate to machine code eventually, but that doesn't make all languages equally easy to understand.

Specifically, some people try to understand code by mentally stepping through it. IMO that's a lot easier in imperative languages than in Haskell. In imperative languages, you step down one line at a time, and loop back in easily specified places. In Haskell, on the other hand, you need to imagine graph reduction in your head. That's already pretty hard for most people, but it gets worse! You can't mentally step through a Haskell function in isolation, because the sequence of graph reduction steps depends on what the caller will do with the answer.

That's not just a theoretical difficulty. Take a look at this thread: [http://lambda-the-ultimate.org/node/3127](http://lambda-the- ultimate.org/node/3127). A bunch of programming language researchers and long- time functional programmers are looking at a six-line Haskell program implementing the Sieve of Eratosthenes, and cannot figure out its big-O complexity. I can't, either.

<comment><USER16> Maybe I am splitting hairs then. I still hold that the _translation_ is fairly simple. On the other hand, the operational steps that the machine code goes through can be mind bending, and I still have trouble figuring it out myself. Perhaps that's what raverbashing meant all along.

</comment></comment></comment></comment><comment><USER17> Clojure strikes an impressive balance between functional purity (which I'll agree isn't always the most natural solution) and mutability. You can write purely functional programs, but you can also carefully introduce state and mutability in a controlled manner.

<comment><USER12> Clojure is my first FP language and I am really enjoying it. Looking forward to introducing it at work.

</comment></comment><comment><USER18> Yeah, _purely_ functional programming can approach an academic exercise. Real- life programs are different, the hosting of mutable state is inherent in many tasks and programs, and cannot be avoided so it makes no sense to trick the language into doing something that it can't easily do.

But functional programming doesn't need to mean all purely functional programming. Functional programming is easy to reason about but it can only be useful as long as it's only applied to the part of the program for which it is a fit. That's quite like how object-oriented programming is a perfect fit for programming user interfaces but not necessarily to managing a data flow.

Write as much in functional style as possible. That will form a nice set of tools upon which you can build. If you need to modify state, you might be able to parameterize a pure function and handle the updates outside but if things get too complex, just pass in a struct or a dict. You can always refactor the function into smaller functions that are, again, purely functions. It's just a matter of drawing the line where it's simplest.

Just as it doesn't make sense to write a program using only global state and gotos it doesn't make sense to write a purely functional program. (And it doesn't make sense to write a program using only objects, either. I'm looking at you, Java.)

There needs to be an interface between the functional code and the code that has side-effects. I like Clojure there a lot. It supports mutable state but makes access to it explicit and controlled. It also support writing a function that looks pure but internally uses a temporary mutable store for data crunching, which is then explicitly finalized before it's returned back as an immutable piece of data.

<comment><USER19> > There needs to be an interface between the functional code and the code that > has side-effects.

Yeah, like some kind of _IO_ type or context. Gosh, someone should try and invent that sometime. :-)

</comment></comment><comment><USER20> I think FP works okay, but at the appropriate scale.

Tell Above, and Ask Below.

[https://news.ycombinator.com/item?id=3876136](https://news.ycombinator.com/item?id=3876136)

When functional code gets unwieldy, it's probably time to place an object layer above it. You see the model in Erlang (where processes act as objects) and the messsaging middleware of many IT systems.

<comment><USER21> It's time for this terrible "Objects in the large, functions in the small" idea to be put to bed.

FP can do side-effects, asynchrony etc just fine, it's just about controlling them so you can still reason about your program and compose its elements easily.

Kay-style message-sending object graphs are neither modular nor composable; state & effects leak transitively by observation, and they cannot be recombined in self-similar patterns. Local reasoning is impossible.

We can talk about specific situations where say, Actors models might have pros and cons, but a blanket recommendation that "it's probably time for an object layer" is rubbish.

Why not design for composability and modularity top to bottom?

</comment></comment><comment><USER22> > You could pass a global state in and out of every function in your program, > but why not make that implicit?

Because making it explicit allows you to reason about what a piece of code does without inspecting every little part of it for the global state it touches.

<comment><USER23> At the cost of having to practicaly rewrite it if the control flow changes even slightly.

If you don't know the exact control flow when you start it's a lof of busywork to modify it each time you change your mind.

<comment><USER22> I'm not sure where you get the idea that much would need rewriting, but if anything does, the advantages of having modular code that you can reason about are worth it.

If you read what I quoted again, he is basically arguing for the use of plain old imperative code over OOP. It's widely considered bad practice to use global state at all in OOP languages, except where strictly necesssary for handling specific resources anyway.

OOP languages still have global variables via "static", but good OOP code design will try to avoid static entirely, and pass any global state explicitly down through functions and classes all the way from main. The reason is simple: It allows you to reason about what every part of code does. The only problem is that you can't reason about some code someone else wrote, as it may use static internally.

And this is really what FP intends to fix. Instead of just "trying" to avoid static, forbid it entirely. Now you can take some code written by anyone and be sure that it isn't going to interfere with another piece of code you write separately. Any time you explicitly want parts of code to interact in such ways, making those interactions explicit still allows you to reason about their collective behavior (and isolate that behaviour from the rest of your code).

Hague suggests that with FP, you need to write in SSA form whenever you're passing state around explicitly, but this argument is nonsense - FP has all the tools to _avoid_ the use of assignment at all. Control flow is abstracted away as functions and they have useful names (or operators). What makes Hague's suggestion most absurd though is his use of an example from imperative languages, where he uses an if STATEMENT. Of course, you probably do need to use variable assignment if your language prevents you treating constructs as expressions.

<comment><USER23> Sometimes it's worth it, sometimes it's not. Exploratory programming is a thing. Example:

Spec 1.0 update position of objects according to velocity

        for in imperative, map in FP

Spec 1.1 what if we made the objects that are far from the center of the group move towards the center?

        for followed by for in IP, reduce and map in FP

Spec 1.2 scratch that, just remove the objects that are far from the center

        for and for in IP, reduce, map, filter in FP

Spec 1.3 and spawn explosions around them

        add 1 call in ip, refactoring required in fp

Spec 1.4 also they should play explosion.mp3

        add another call in IP, another refactoring required in FP

Spec 1.5 this should happen for all destroyed objects

        move both calls to function that removes object in IP, another refactoring (with passing a lot of state in new ways) in FP

And so on.

After you're OK with the result you can make the code nice in IP too. And you can do this once, instead of doing this every time anything changes.

It's like a text editor that forbids you to have temporarily non compiling code. Only refactorings are available, you can't write if by hand. Pain in the ass.

Yes I know about paredit, people still copy paste, and write parents by hand.

<comment><USER22> It'd be nice if 1.3 and 1.4 could be done with just 1 line of call, but you're missing the fact that pretty much any graphics and audio library will have some "context" singleton through which it is called due to the way these interact with the operating system. The APIs are deliberately designed this way to encapsulate the global state they use.

Sure you can make a global variable for the context and avoid passing it around - but the refactoring route is the preferred option in the imperative world anyway.

If you're writing in Haskell with any expectation of using IO for some exploratory programming like this, the obvious solution is to make your functions of type (:: MonadIO m => ... -> m ...), then any refactoring you need to do to add additional facets to your state can be specified elsewhere as part of a MonadIO transformer. Your calls to "play explosion.mp3" and "render explosion" just need a liftIO on them.

</comment></comment></comment></comment></comment><comment><USER24> > Make those globally accessible and modifiable and all of a sudden a large > part of the code has shifted from imperative to functional

Isin't it the other way round?</comment>


Canada's top court backs order for Google to remove firm's website from searches - <AUTHOR> http://www.cbc.ca/news/canada/british-columbia/google-ruling-1.4181322 <comment><USER2> Tremendously impactful decision, regardless of which side of the case you support.

Interestingly, the majority addressed a Google argument centered upon concerns regarding the possibility of international censorship:

"Google's argument that a global injunction violates international comity because it is possible that the order could not have been obtained in a foreign jurisdiction, or that to comply with it would result in Google violating the laws of that jurisdiction, is theoretical. If Google has evidence that complying with such an injunction would require it to violate the laws of another jurisdiction, including interfering with freedom of expression, it is always free to apply to the British Columbia courts to vary the interlocutory order accordingly. To date, Google has made no such application. In the absence of an evidentiary foundation, and given Google's right to seek a rectifying order, it is not equitable to deny E the extraterritorial scope it needs to make the remedy effective, or even to put the onus on it to demonstrate, country by country, where such an order is legally permissible."

In other words, if an interlocutory order with international scope would violate foreign freedom of expression legislation (or other legislation in general), it would be possible to seek to vary the interlocutory order by raising those issues specifically.

This framework should be familiar to jurists in Canada, as it resembles the Paramountcy doctrine.

Whether or not this case will be widely used is unclear. This case is nearly 100% fact-perfect for the Respondent - it honestly looks like a civil-rights test case. It is very possible that imperfect everyday facts provide sufficient fuel for judges to distinguish this case from the case in front of them.

The dissent is interesting because it lists a number of pieces of evidence the dissenters would have needed to see before moving forward, including the impact of the first order on sales figures. These dissents often provide counsel with information regarding how to structure future cases in order to avoid outstanding concerns.

These are my first pass thoughts. I'll probably read it through another two times before the day is done.

<comment><USER3> Should every court and country have the right to demand that things are de- indexed worldwide, now that Canada has done so? Does China have the right to demand Tiananmen Square be de-indexed? Germany and Nazi symbols? Russia and satire about Putin? North Korea? The United States? If not, how do we decide?

The previous framework was, "you can demand whatever you want within your borders; if it is unacceptable, Google will leave." This decision throws a bit of a wrench into that.

<comment><USER2> I touched upon this, but honestly this is a very, very large subject that requires familiarity with the Berne Convention and TRIPS/ACTA in order to fully explain.

In the Canadian decision, the de-listing order is contingent on that order not being contrary to compliance requirements elsewhere in the world. However, the SCC doesn't have knowledge of all of those laws, nor should it. Accordingly, it stated the burden upon showing a contradiction rests with the applicant, Google, who didn't want to comply with the order. Google would need to cite specific laws they were in contravention of, rather than just pleading that there may, maybe, somehow be a contradictory requirement. What if there wasn't? Should Google still get to plead international comity in that instance? No.

Positive freedom of speech legislation might work as an alternate requirement, but this was not plead, and this was not a situation where the speech in question is political or particularly protected. It is a very clear instance where a supplier has fled Canada (and they can't find where they're based) in order to attempt to continue crystal clear trademark violations which are causing actual damage to a Canadian business.

The current international trademark framework means that the majority of nations in the world would actually require compliance with this order (well, not exactly, but close enough). The big question is how that machinery should be activated. If the Respondent had the money to pursue passing-off litigation in every jurisdiction, they would likely arrive at a similar result, but functionally, a small company won't be able to pay x200+ times the legal fees for a similar result while their dealflow is being sapped.

This decision, even read very broadly, wouldn't provide countries with the ability to de-index items. Countries don't need that ability; they already have it. Most, however, have bound themselves not to exercise it, but certain, like China, actively pursue control as an overt element of their policy agenda.

I feel like I'm doing a bit of a disservice to the topic, because there is a lot of small nobs and levers that I can't get into without exploding this into a 20 page memorandum. I am confident in stating that the freedom of speech concerns arising from this case are not as dire as portrayed.

Edit: Made the middle a bit clearer.

<comment><USER4> Can you help me understand something. Why do they have to cite laws this ruling is in contravention of instead of citing laws that Google is in convention with to prevent worldwide dissemination? One would think that you don't have to prove what the government proposes you do in region X to be illegal there, but rather prove that what you are already doing in region X to be legal there.

<comment><USER2> The Supreme Court did cite the test governing the provision of interlocutory orders. That test is well known law from _RJR MacDonald_. There's a bit of a blurb regarding the case in my comment below, but you can look up the case and you'll find the full test and commentary.

The government isn't proposing anyone do anything; A private party is, and asked for the court's help. The second private party is very clearly acting illegally and appropriating the value of the first party's brand. They've gone as far as to flee the province to try and dodge the court order which attempts to halt them from doing so. The first party asked google to take down their listings because those listings are the channel through which the illegal brand appropriation is occurring. The court agreed.

Google said they didn't want to comply with the order partially on the basis that there might be some compliance contradictions. The court said "Sure, but you didn't plead any evidence that that's the case. We judge what's in front of us. If you're serious, we have a mechanism to deal with it. Make an application like anyone else who wants to amend an interlocutory order. Bring us evidence there's another law which prohibits you from taking the listing down, and we'll amend the order".

The reason why courts only judge what's in front of them is so that the other side can actually argue against their points. If Google said "Law X from Country Y prevents us from doing that", the Respondent could reply "The only thing Law X from Country Y does is regulate how sweet twinkies can be, please stop lying to the court". The alternative, forcing the Respondent to prove that there is NO contradiction in every other country, is impossible. Google's response, however, requires 1 list distribution email in legal saying "Anyone have a problem in their country? Let me know". This is the "balance of convenience" portion of the _RJR MacDonald_ test.

<comment><USER4> The unfortunate part of that test appears to be that the private party can request the moon. A reasonable person might expect a court that is required "judge what's in front of them" to not apply their judgement to foreign jurisdictions.

I wonder how this would be applied to a physical storefront from a global retailer that happened to have presence in Canada...I suppose they'd be forced to remove physical non-Canadian products from even their non-Canadian stores. Because hey, retailer, show us where it's illegal for you to remove this product. You can't? Then apply our ruling worldwide.

Is my analogy off? Or is it, as I suspect, the fact that the decision is easier to accept digitally part of the reason why it's being levied? The RJR MacDonald test appears incredibly subjective.

This test, if in fact it is narrowly construed as show-us-where-you-can't, is a scary defendant test to counter decisions. Or if the test is being vaguely construed for these means, it is equally scary.

<comment><USER2> Private parties cannot request the moon. The test for interlocutory orders is HARD to satisfy, and involves substantial expense for litigants to raise. In Canada, if you lose this motion, you're paying half your opponent's legal bills most of the time, too.

You need three things to satisfy the test:

1) a serious issue to be tried 2) irreparable harm if the injunction is not granted 3) that the relief is just on the balance of considerations

Note that irreparable harm and balance of considerations are terms of art, both of which heavily restrict the ambit of what is admissible under those sections of the test, a process which is continually monitored by the development of subsequent jurisprudence.

But even if the injunction test is satisfied,

The physical products hypothetical you're advancing is incomplete and leaves out a number of facts which are instrumental in the court's ruling. Assuming the ruling is applicable outside of those parameters requires that clear arguments be made regarding why changing those parameters isn't a big deal.

As an exercise, try to go through the ruling and pull out each of the factual elements which the court seems to rely upon in coming to their decision. You'll note they rely on a lot of determinations to get to the decision they ultimately arrive at.

Other replies have gracefully dealt with the physical goods analogy more directly: things get ripped off shelves all the time. That's why the Apple v. Samsung cases are so high-stakes.

<comment><USER4> It is not the previous, original decision that affects Canada only that I am concerned about. It's the global reach here. Some of their wording about Google inadvertently assisting the non-Canadian company outside Canada strike me as not understanding how the web works. I concur with the dissenters that mentioned the issue should be taken up with the other company's government (France?). It just reeks of protectionism, applied outside the country's jurisdiction because they have the leverage to do so.

> That's why the Apple v. Samsung cases are so high-stakes.

It would be like the US court told Samsung they couldn't violate Apple's US patents on phones sold in Mexico.

</comment></comment></comment></comment></comment></comment></comment></comment><comment><USER4> A hypothetical: Company A is in country X and company B is in country Y. Country X declares on behalf of company A that company B be blocked from searches worldwide (including country Y). Country Y declares on behalf of company B that company B cannot be blocked from searches (at least in country Y). Who wins in the global context? Whichever can/will fine the most? Does this give benefits to the overly-regulating country/region and is that what we want?

Another hypothetical question: Can Canada require some retailer in the US to remove another non-Canadian company's products in the US just because the retailer and a competing company both have presence in Canada?

<comment><USER5> The bigger question is if Canada can enforce their regressive idea of Free Expression on US Service providers simply because of their presence in Canada

Canada has no concept of the 1st amendment,Canada as all kinds of limits on Speech that will never be allowed under the US Constitution.

<comment> 52-6F-62 Canadian here. Would you be so kind as to elaborate on your sweeping allegation regarding free expression? As it is, this just seems like some random stab that doesn't concern the topic at hand (which has far more immediate concerns tied to it).

Section 1 of the Constitution of Canada (1982) outlines freedom of speech. Just about the only thing you can't do, outside of pursuing damages in the court system for specific cases, is stand on the street screaming "Let's kill all white/black/asian/aboriginal people right now". For that, you might land yourself in some trouble.

<comment><USER6> How do you square the Constitution of Canada with the Human Rights Commission that fines comedians and authors?

[https://en.wikipedia.org/wiki/Canadian_Human_Rights_Commissi...](https://en.wikipedia.org/wiki/Canadian_Human_Rights_Commission_free_speech_controversy)

<comment><USER7> It's actually a great example of how freedom of speech is not a binary. It's possible to curtail some forms of expression, such as hate-speech, without turning the country into an autocratic regime that punishes thought crimes. Lots of Americans like to think that as soon as you moderately curtail freedom of expression, you've crossed some threshold and you're now North Korea. Turns out that with a strong judiciary (that isn't corrupted by money and politics) it's possible to have reasonable constraints on expression!

<comment><USER5> Except Free Speech is a Binary, and that is exactly what Canada is doing,... punishing thought crimes.

>Lots of Americans like to think that as soon as you moderately curtail freedom of expression, you've crossed some threshold

You have crossed a Threshold

>and you're now North Korea.

Nice strawman you have there.....

>it's possible to have reasonable constraints on expression!

Proving you do not have or support Free Speech, you support "Reasonably Restricted Speech" which is not the same thing

<comment><USER8> > Except Free Speech is a Binary, and that is exactly what Canada is doing,... > punishing thought crimes.

A "spoken crime" is not a "thought crime". There's no chip in your head, you're free to think whatever you like, even if you're not necessarily free to say anything you like.

<comment><USER9> You cross the line into thought crimes when you ascribe intent and presume to know the state of mind and thought process behind a given piece of speech.

<comment><USER10> Hmm, do you think there are some things you can say that would classify as an action you could be arrested for rather than an idea?

For example, yelling "Fire!" in a crowded building, offering a bribe, and threatening someone all seem like actions you could be arrested for in the USA where the only evidence is your speech (or a recording of it). Would these qualify as thought crimes to you and if so do you think there is a developed nation with free speech where they are accepted?

<comment><USER9> How about threatening violence against a member of a protected class? Is the crime worse because of that persons protected status, even if that didn't matter to the perpetrator?

<comment><USER5> There should not be protected classes..

All people should be treated the exact same under the law

</comment></comment></comment></comment></comment></comment></comment></comment></comment></comment></comment><comment><USER11> This wouldn't be the first time a government is requiring Google to modify their search results. This case parallels the "right to be forgotten" cases brought against them by the EU several years ago. There is a reason that civil liberties and human rights organizations like the ACLU are concerned about this precedent.

In the US, with the murders of unarmed civilians by law enforcement and subsequent acquittals occurring at an alarming rate with increasing public outrage, it might be only a matter of time before a court somewhere rules in the favor of a person found innocent who is suing to keep as many details of a particular murder off the Internet on the grounds that his or her constitutional rights being violated (i.e inability to find employment, friendship, etc), and companies such as Google being forced to comply with the court's rulings.

<comment><USER12> > it might be only a matter of time before a court somewhere rules in the > favor of a person found innocent who is suing to keep as many details of a > particular murder off the Internet

Which in the context of "murders of unarmed civilians by law enforcement and subsequent acquittals" sounds bad, because it's possibly against your concept of justice, but what about the other end of the spectrum, where baseless claims are leveled on a person and this is high in many search results, or people are defamed through libel and the jurisdiction doesn't allow allow for a legal remedy?

I mean, if I seed the internet with 20 SEO optimized sites about how you're a murdering rapist, should Google's right to keep that in results be defended? Freedom of speech is complicated, and _not absolute_ in the United States. We have laws for libel and slander to prevent just this sort of thing, and I don't usually see those being called a bad idea. Perhaps that should be an indication that this issue isn't as black and white as it's often portrayed, and deserves a more moderate, considered solution.

<comment><USER11> > but what about the other end of the spectrum, where baseless claims are > leveled on a person.

You are confusing imaginary baseless claims with matters of public record. The potential danger of abuse is in allowing individuals/entities using the law to keep potentially unfavorable public records off from the Internet as a matter of public good, as with the right to be forgotten cases in the EU.

<comment><USER12> > You are confusing imaginary baseless claims with matters of public record.

So your assertion is that no baseless claim can make it to court only to be defeated or dismissed later and become part of the public record?

> The potential danger of abuse is in allowing individuals/entities using the > law to keep potentially unfavorable public records off from the Internet as > a matter of public good, as with the right to be forgotten cases in the EU.

That's _one_ potential danger. Let's not lose sight of other potential dangers just because we're focused on one that's getting a lot of attention in the moment.

<comment><USER11> > Let's not lose sight of other potential dangers just because we're focused > on one that's getting a lot of attention in the moment.

Sure, the Cosby trial might be an example of another danger. He was found not guilty despite claims that were ultimately found by a jury of his peers to be baseless or questionable.

But this really isn't about the spectrum of abuse of allegations, but rather the danger in how someone may legally try to suppress publicly available information because of a particular court deeming it in the public interest to.

<comment><USER13> You are confusing a mistrial with a not-guilty verdict.

<comment><USER11> Considering that the DA threw everything but the kitchen sink at the Cosby case, a retrial would be either unlikely or unfruitful, given the judge's intentional leaking of juror's names to the press and hinging his judicial career on a guilty verdict prior to the trial, a second trial could result in legitimate claims of malicious prosecution, making any conviction easy to appeal and ultimately overturn by a higher court.

<comment><USER13> The only thing that we have right now is a mistrial based on a hung jury, no more, no less.

What the judge did (which I agree was despicable) and what the DA did (which doesn't matter) has no bearing on whether or not there will be a re-trial in 4 months. Cosby will appeal if he's convicted so from that point of view you could skip the trial entirely, his best bet is that he may be found not guilty.

</comment></comment></comment></comment></comment></comment></comment></comment><comment><USER14> Canadian law professor Michael Geist has a pretty good summary/analysis of this case:

[http://www.michaelgeist.ca/2017/06/global-internet- takedown-...](http://www.michaelgeist.ca/2017/06/global-internet-takedown- orders-come-canada-supreme-court-upholds-international-removal-google-search- results/)

</comment><comment><USER15> The EFF is apparently on the side of ElGoog:

[https://www.eff.org/cases/google-v- equustek](https://www.eff.org/cases/google-v-equustek)

"Such a broad injunction sets a dangerous precedent, especially given that it is likely to conflict with the laws of other nations."

</comment><comment><USER16> > "Today's decision confirms that online service providers...have an > affirmative duty to take steps to prevent the internet from becoming a black > market."

Well that's terrifying.

</comment><comment><USER17> > Google voluntarily removed hundreds of webpages from its Canadian search > results on Google.ca. But the material continued to show up on Google's > global search results.

> So Equustek obtained a further injunction from the court ordering Google to > remove the websites from its global search results.

> Google appealed and argued it was not a real party to the dispute, and that > a global injunction would violate freedom of expression.

Google didn't object in principle to removing the listings from google.ca but did object to removing them from the the main site results. What does that mean, there's actually no objection in principle but there's enough technical challenge or cost to modifying the global results that they're willing to fight it in court?

<comment><USER18> There's a question of jurisdiction. Should any country's courts be able to restrict what happens in other countries?

<comment><USER19> I don't know if they should, but de-facto they can. American companies are, for instance, forbidden from doing business within Iran, North Korea, etc. Likewise, nothing stops the US from passing an enforcing a law that requires any foreign companies that do business with the US to follow US laws, even in their operations outside the US.

<comment><USER20> My mental model for this is as follows:

1\. If you are a company based in and operating as a legal entity in Country X, the laws of Country X apply to your business as a whole, including services you offer in Country Y and Country Z.

2\. If you are a company based in and operating as a legal entity in Country X but are offering services in Country Y, the laws of Country Y apply only to the services you offer in that Country Y.

All of this can then get mushy and complicated based on international governing bodies, multinational trade agreements and nation-state soft power but absent those constraints, I believe this holds.

</comment></comment></comment></comment><comment><USER21> I've never expected to use a Russian search engine to evade censorship but there we go:

[https://www.google.com/search?q=datalink+gw1000](https://www.google.com/search?q=datalink+gw1000)

[https://yandex.ru/search/?text=datalink+gw1000](https://yandex.ru/search/?text=datalink+gw1000)

<comment><USER22> I'm amazed they provide links to other search engines at the bottom of the results.

<comment><USER21> I suppose it's for convenience when yandex doesn't find what you are looking for. Don't worry, Russia isn't exactly a freedom of speech heaven and if the government had a serious issue with these other search engines they wouldn't be there.

[https://en.wikipedia.org/wiki/Federal_Service_for_Supervisio...](https://en.wikipedia.org/wiki/Federal_Service_for_Supervision_in_the_Sphere_of_Telecom,_Information_Technologies_and_Mass_Communications)

<comment><USER23> At the same time you don't see google or bing linking to other search engines. So I think linking to the competitors for your users convenience is a pretty neat thing to do, which probably irks countless top managers daily. Don't belittle that little neat touch just because it's a russian website.

<comment><USER21> True and I don't dispute that.

I'm not even sure if what I posted makes much sense, I was somehow stuck in the context of using alternative search engines to evade censorship and figured that parent suggests that these links may have something to do with it.

</comment></comment></comment></comment></comment><comment><USER24> >"We have not, to date, accepted that freedom of expression requires the facilitation of the unlawful sale of goods."

That precise argument may be well-intentioned, but it threatens free speech because it sets a precedent placing a burden of acceptable effects and results of free speech.

In other words, should Tiamen Square be de-indexed globally because free speech in China does not require it. Should torrent trackers be de-indexed-- free speech does not require illegal file sharing, after all? What about bit torrent clients? Tor browser?

This site may or may not be rightfully de-indexed, but it is not because of some limitations of free speech.

</comment><comment><USER25> I was expecting the ruling to involve some really bad behavior such as selling misslabeled drugs or invading the privacy of private individuals. Instead it is a benign product relabling suit. IMO, product relabeling doesn't even harm the company who's products are relabeled.

<comment><USER26> Could you elaborate? I don't really know much about relabeling. The company seems to be claiming that the infringing company is selling products under company A's name when they have no actual affiliation, which sounds like ordinary fraud to me.

edit: Nevermind, I misread. There is no fraud or impersonation occurring.

<comment><USER25> If the article is correct, I quote "Equustek had successfully sued another company for relabelling its products and passing them off as their own.". That means that the offending company was buying Equustek's product and putting a different sticker on them.

<comment><USER26> Ah, thanks. I parsed that sentence incorrectly the first time I read it.

</comment></comment></comment></comment><comment><USER27> Why don't they just use robots.txt to keep themselves out of Google's index? User-agent: Googlebot Disallow: / Has Google stopped using that directive?

<comment><USER28> It wasn't their site. From the article:

"Equustek Solutions based in Burnaby B.C., which manufacturers networking technology. Equustek had successfully sued another company for relabelling its products and passing them off as their own."

When the evil corp lost in Canada, it moved out of the country and continued the same practice.

Frankly, I'm disappointed in Google for fighting this in the first place. SMH. Imagine if this were Apple rebranding Samsung Galaxies as Apple devices. Of course Google should remove those product listings.

<comment><USER29> Why can't you rebrand a product? You bought it, you own it, you can resell it.

This case is much more complicated however. It's two business partners who had a falling out and both tried to put each other out of business. The marketing guy kept their own brand and customer base while the tech guy tried to rebrand and sell his products on his own. The marketing guy hired an engineer to make a new product that does the same function. The tech guy accuses him of copying his technology.

[http://robertfleminglawyers.com/pdf/2012_BCSC_1490_Equustek_...](http://robertfleminglawyers.com/pdf/2012_BCSC_1490_Equustek_Solutions_Inc._v._Jack.pdf)

<comment><USER30> You're kind of missing the bit about the engineer they hired used to work for the other company.

</comment></comment></comment></comment><comment><USER31> I wonder if the website in question is listed on other search engines (I can't see why not).

Assuming this is the case, is it fair to force one company to expend the effort altering their search results when others don't have to? Did Google do anything wrong that Microsoft (Bing) or Yahoo didn't?

</comment><comment> awinter-py Ironically, now that G has been ordered by a court to take the content down, keeping it up is a form of protest, i.e. a comment on the law, i.e. political speech.

<comment><USER32> No irony to it, freedom of speech is way more limited in Canada: [https://en.wikipedia.org/wiki/Freedom_of_speech_in_Canada](https://en.wikipedia.org/wiki/Freedom_of_speech_in_Canada)

My journalism teacher used to say that legally speaking, there is no such thing as free speech here.

Either way, I'm sure the Supreme Court has considered this.

<comment><USER19> There is plenty of free speech in Canada. It draws the line at hate speech differently then the United States does, but the concept is not foreign to the US either. See - the checkered history of 'fighting words'.

</comment></comment></comment><comment><USER33> This kind of stuff should be arbitrated in an international court. Giving an authority in a single country the say on what can and can not be seen on the internet world wide is a terrible idea.

<comment><USER19> Under which legal framework? Common law? Civil law? Old Testament law? Sharia law? The framework laid out by the Constitution of the Soviet Union? The US codex of law, circa 1953? Circa 2007? Circa Citizens United?

Should everyone get a chance to vote for choosing it? Should we decide based on population of respective countries? Or do a one-country-one-vote thing? Maybe one dollar of GDP, one vote? Why should Chinese commercial law have any bearing on a trade dispute between Honduras, Belgium, and Spain?

<comment><USER34> Multi-national binding protocols have been done in the past and will be done in the future. Where the international community sees it necessary, there can be the creation of international law. Certainly this case is something for which international law and enforcement regimes can be created.

<comment><USER19> They are all the products of bi-lateral or multi-lateral treaties, hammered out in secret meetings between senior members of participating governments.

There is absolutely no "International community" involved in this process. It's a private settlement between directly affected parties.

These agreements also tend to be incredibly undemocratic, and are often hilariously hostile to the interests of the constituents of said governments. Be careful what you wish for - what you'll get is less like the European Union, and more like the TPP.

</comment></comment></comment></comment><comment><USER35> Every time something like this happens I wish YaCy ([http://yacy.net/en/index.html](http://yacy.net/en/index.html)) was in better shape.

<comment><USER36> I wasn't aware this existed but I am absolutely intrigued.

</comment></comment><comment><USER37> Not honoring [http://www.robotstxt.org/faq/prevent.html](http://www.robotstxt.org/faq/prevent.html) can be a felony;

</comment><comment><USER38> If a product or service is a proven scam, it is highly likely to be a scam in all jurisdictions. Googles own takedown procedure should have fixed this before it got to the courts.

</comment><comment><USER39> Completely read that wrong. Thought it order google to remove it's own website addresses from search results. that would have been interesting.

<comment><USER40> This would be a significant victory against people who use Google to search for Google and then perform a search. These types do exist.

</comment></comment><comment><USER41> Removing results from google makes so little sense.

Can I buy google ads pointing to those blocked websites?

</comment><comment><USER42> I wonder if this decision can be used as a framework to kill extortion-based sites such as RipoffReport.com that Google has aided and abetted for more than a decade. That would be wonderful.

<comment><USER9> >aided and abetted

I'm not sure that phrase means what you think it does... Or perhaps you simply misunderstand what Google does?

<comment><USER42> I realize that you were trying to be condescending, and you did a good job of it. However, Google has rewarded this site in particular with staggeringly high search rankings, which makes things especially sticky for the victims of the site. They have been on notice about this site and others like it for almost the entire time that they have existed [1], and have actively refused to even adjust their rankings - despite clear evidence that these sites are committing extortion. So yes, Google is aiding in the commission of the crime of extortion.

If I were to file a complaint on the site about how someone named "djrogers" is a condescending jerk on HN that broke into my house and stole my wallet, that complaint would be posted on the site within minutes with no fact checking, and then Google would rank it almost instantly at the top of their results for "djrogers". One day you might Google your username and see this complaint in the #1 position, and you would probably contact the site saying that this is absolutely false. The site's owner, Ed Magedson, would personally respond essentially saying that he has been sued many times, that he has always won, and that the report isn't going anywhere...unless you pay thousands of dollars (allegedly the fees go to "investigate" the veracity of the claims in the report, after which it will be amended or removed if found to be entirely false - but only if you pay). He will make extensive reference specifically to how well his site ranks in Google, with the implied threat that if you do not pay, anyone who Googles your name will see this defamatory information (which they will).

When you know that you are assisting in the commission of a crime, you are aiding and abetting.

[1] [https://moz.com/blog/the-anatomy-of-a-ripoff-report- lawsuit](https://moz.com/blog/the-anatomy-of-a-ripoff-report-lawsuit)

</comment></comment></comment><comment><USER43> Google should have country specific feeds (canada.google.com) instead of domains. Then location based filtering is forced on the browser where it should be.

<comment><USER44> I'm not sure I follow - canada.google.com _is_ a domain, as is google.ca.

What are you actually suggesting?

</comment></comment><comment><USER45> This is total bullshit. Relatively benign ruling but sets a dangerous precedent.

<comment><USER19> For which courts? Canadian rulings would not be considered precedent-setting in a Saudi court.

<comment><USER46> Its precedent settings i.e. if google follows it, then china and saudi court can ask them to do the same since these kind of rulings become fair game.

<comment><USER47> Chinese and Saudi courts can do that to the extent the local law allows them to independent of whether Canadian courts do; Chinese and Saudi courts are neither bound nor empowered by Canadian precedent (I doubt either would even treat it as _persuasive_ precedent, much less binding.)

</comment></comment></comment></comment><comment><USER48> I am confused as to how a Canadian court could have any sway on search results in other countries? Is there even any legal precedent on the matter?</comment>


Announcing my bootstrapped startup: RestBackup - <AUTHOR> http://www.restbackup.com/

<comment><USER2> It looks interesting, but at this point it feels more like a "web app" than a "startup". Granted, the two are often interchangeable these days, but to me the difference is one of formality and commitment.

For example:

What happens to my data if you decide that this isn't fun anymore? What happens if you get sign-ups that store a bunch of data but don't pay their bills? What happens if you get sued, are you personally liable or is this an entity? What are the terms of use? What kind of support is available?

From your site, it appears that you are in the process of solving these questions, but there's likely to be a lot more road to cover before it goes from web app to startup in my mind.

Good luck!

<comment><AUTHOR> Yes, these are details that every new company must attend to. I will accomplish these in the coming months. Let me answer a couple of your questions:

1\. I'm doing this to make money. I also want the satisfaction of creating something of value to folks all over the world. It's high-time that our software back itself up automatically.

2\. When a customer uploads a file to RestBackup, their account is billed for the cost of storing the file for the full retention period. If they don't pay their bill, their account will be deactivated, but their paid data will be kept. Unpaid data will be handled on a case-by-case basis.

3\. I will choose a lawyer this month to iron out the terms of service. Incorporation will occur before the paid Beta-test program begins.

There's an even more important question: Am I building the right product? I need early adopters to help answer this question. Thanks for your feedback and I hope to see you in the RestBackup alpha test!

</comment></comment><comment><USER3> Technicalartup point of view - what value are you adding, and what are you doing that someone else can't duplicate with a minimum of effort?

Right now this is a fairly transparent re-use of S3 with some clever features on top - I can see the appeal - but I'm thinking "Yeah, we could rig that up over the weekend and do the same thing here.... no need to pay double the price for S3 storage when a few hours of in-house work will take care of it.

Also - as to "never delete" "Never" is an awfully long time. Mistakes are made, and while "never" might seem like a safety net, there also might be a time when I _need_ to delete data that's been stored offsite -for legal reasons or because someone screwed up.

It looks like something targeted at technically-savvy users, and those users are probably equally capable of doing this themselves with S3 or otherwise.

<comment><USER4> Regarding your point about this being something easily duplicated, I think the point of this service is that it frees you of the need to worry about building and maintaining a non-core activity. People could also easily clean their own house and cut their own grass, but millions of people hire others to do the work.

<comment><USER5> I don't think this applies in this case, those activities take time at constant intervals. This saves some time once in implementation for a constant additional cost.

</comment></comment></comment><comment><USER6> What does this service do that can't be easily done by a client-side library? Also, S3 already has a REST API (see <http://mashupguide.net/1.0/html/ch16s05.xhtml> ), you've just eliminated the URL signing bit.

<comment><AUTHOR> RestBackup keeps every application's data separate. Data uploaded through an access url can be downloaded only through the same access url. This is super useful for commercial software, since each user and license can have its own access url. To accomplish the same thing with S3, one would need to run a request signing service.

RestBackup also prevents accidental and malicious deletions. The service keeps your data for the specified period of time no matter what.

<comment><USER7> Interesting points, so I would suggest emphasizing those more. Right now your page jumps right into API level stuff like URL's and curl commands. If I'm just glancing at your site nothing stands out as to why it's different than S3. I need to be sold on it before I ever get to the point of using those commands.

</comment></comment></comment><comment><USER8> Would you tell us more on how you are going to bootstrap the company? I mean, with no VC money involved, are you investing yourself in the EC2 instances/space that you need in advance? Do you have a day job?

Good luck!

<comment><AUTHOR> I worked at Amazon as an SDE on the SimpleDB team for a few years and saved my runway money. At Amazon, I learned a lot about running highly scalable web services. I left Amazon a month ago to build RestBackup full-time. I'm taking full advantage of AWS to keep expenses low and grow with revenue.

Now I need early adopters to help me turn this into a killer product!

Thanks!

</comment></comment><comment><USER9> This is excellent. I love the simplicity of it.

I also love it that files can't be overwritten - clearly you thought this trough! I am currently appending large random strings to my S3 uploads to avoid overwrites.

However, I would prefer to have the files stored in my own AWS account. I'm thinking I would create a special bucket, give you the credentials for it and you would run the EC2 instances required to accept the files and store them there. You would bill me per URL and for the bandwidth, while Amazon will bill me for the storage directly.

<comment><AUTHOR> I'm glad that you like the API. :) I did think a lot about it, but it still needs a lot of work.

Why would you like to use your own S3 bucket as the backing store? I would love to talk with you more about this.

<comment><USER9> Because I want to retain ownership of the data. Transmission is transient and I'm ok with outsourcing that to a little-known company, however storage is lasting and I want to retain meaningful control over it.

I understand that ties you to AWS and restricts your future options, but this might be a good place for you to start - targeting existing AWS users who need a little bit of extra automation.

</comment></comment></comment><comment><USER10> Not sure if I missed it in the API documentation but what happens if I PUT a file with a file name that already exists?

One thing that drives me nuts about a lot of backup services is that they say that the data is encrypted. It is important to recognize the two places where data can be intercepted:

* In transport, unless it's done over TLS/SSL. Most services provide this. * Once stored. This is done very rarely and if it is done, the provider is the one that has the encryption key.

With your service you can actually take care of both by simply instructing your users to encrypt the file before uploading it.

Another bits of functionality I'd say you are missing are "diff" backups. I don't necessarily want to upload my entire 300GB multimedia library each day when I want to back it up (especially if I'm being charged for traffic). I'd rather upload a small diff.

A very cool use of this technology would be a FUSE plugin for browsing your storage area.

<comment><AUTHOR> Hi Igor, thanks for your comment. If you PUT a file that already exists, you'll get a 405 Method Not Allowed error. With the right software, you could keep track of what is already uploaded and compute a diff locally on your computer. When uploading a diff, you would want to extend the expiration of the previously uploaded files. I'm open to suggestions for how to add this to the API.

The httpfs FUSE plugin should work with RestBackup. I can fix it if it doesn't work. Would you like to sign up for the alpha-test and give it a try?

</comment></comment><comment><USER11> \- I love clean REST APIs so I can clearly see the value in your service over others (e.g. using S3 directly). But the benefits probably aren't immediately obvious to most potential customers. Maybe consider a side-by-side comparison of steps involved.

\- Do you encrypt the data before storing it to S3 or store as-is? I need to think about it more but assuming you store as-is, I think I prefer that over Tarsnap's approach. You may want to consider a comparison between this service as well. (As a side note, I would be using Tarsnap right now but I'm in Canada)

\- What is the need for charging for Request Processing? I guess that would prevent storing many small files, but it complicates the pricing model. It will be easier to communicate your offering if all costs are included in the data transfer and storage prices.

<comment><AUTHOR> I'm glad that you like the API. I'm working on a side-by-side comparison of RestBackup and other services.

RestBackup receives the data from the customer via HTTPS and submits it to S3 also over HTTPS. So the data is encrypted in-transit but remains in S3 as-is. This is the minimum viable product. I'm considering encrypted storage as a feature for the public Beta.

RestBackup has per-request charges for the same reason that S3 and CloudFiles have them: to avoid losing money on high-volume request loads on small files. If you need to store many small files, it might be more efficient to package them up in a tar archive.

Thanks for the great comment and the tweet! I hope to see you in the alpha- test!

</comment></comment><comment><USER12> _> Use the HTTP PUT verb to upload your files to the service. Since this is a REST interface, the file will live at the path you specify. Retrieve it with an HTTP GET. You cannot replace existing files. PUTs to existing files will fail with "405 Method Not Allowed"._

PUT does not mean what you think it means. The whole point of PUT is that you're replacing an existing resource, which gets you idempotence. You're not supposed to use it to create a resource at the supplied URL, and the whole point is that you can PUT to the same URL repeatedly.

You want POST. Using PUT is making your service _less_ restful.

<comment><AUTHOR> I'm pretty sure I got it right. The HTTP 1.1 RFC says:

> _The PUT method requests that the enclosed entity be stored under the > supplied Request-URI._

<http://www.w3.org/Protocols/rfc2616/rfc2616-sec9.html#sec9.6>

I interpret this to mean that if you PUT to /some/file then you should be able to GET /some/file and get back the same thing. RestBackup access urls provide this behavior.

<comment><USER12> Sorry, you're right about using PUT to create a resource at the supplied URL -- but you're still breaking the expectation of idempotence: repeating the same request should have the same result, not 201 the first time and 405 thereafter. If you really want those semantics use POST.

If you're really set on using PUT, you could change your semantics a bit so that PUTting the original file again results in 200 OK but attempts replace it with a different file return 409 Conflict or something.

</comment></comment></comment><comment><USER13> what are your thoughts on the high cost of S3 vs. the cost of the infrastructure involved? Is your plan to just target customers for whom the premium doesn't matter? or do you plan on using your own infrastructure at some point?

<comment><AUTHOR> I'm starting out using my Amazon work experience to make efficient use of S3, SimpleDB, and EC2. But I am definitely open to lower-priced products built on other services and our own infrastructure. I'd love to talk with you about what your customers need.

<comment><USER13> my customers need

1\. cheap. (these are my customers, right?)

and

2\. Not managed or controlled by me. (the whole /point/ of backups is to cover them if I really screw it up, so backing up on to my own equipment is of limited utility.)

Actually, a s3 clone at half the price would be about perfect.

</comment></comment></comment><comment><USER14> How are you going to compete when there are so many free storage services? A company I work for is contemplating how to provide this kind of in-application backup, but with so many free options - Google Docs storage, SkyDrive, DropBox etc., I wouldn't be likely to opt for something that required payment. What's going to be your differentiator?

<comment><AUTHOR> Thanks a lot for your question. I would love to hear more about your company's needs.

Free services have their limitations. If you want to ship a product that can backup out-of-the-box, without any customer configuration, good luck doing that for free. Save yourself the pain and use RestBackup. My management API lets you create and manage thousands of access urls, with per-url limits. A software vendor can use this API to generate an access url for each application license that they sell. This way each customer's data is separate and secure and the vendor maintains control of the charges incurred by each customer. Paid services like RestBackup can offer these kinds of management features while still remaining economical for commercial software vendors.

I'm not interested in competing with free services. I want to build a product that folks will pay for.

</comment></comment><comment><USER15> Also, what's this with the width of the page being > 1280 pixels? I thought 1024 is still the common standard.

<comment><AUTHOR> Can you recommend a good website designer who can fix it for me and make the site look great?

<comment><USER15> I found the site design quite ok. The logo needed a bit of spicing up to convey what you do, but other than that the site was clear at laying out what your service was all about. The width was the only issue that caught/poked me in the eye.

</comment></comment></comment><comment><USER16> I like the MVP feel. No delete is a great example of "we'll build it later," in my mind.

<comment><AUTHOR> Yeah, my first draft of the API was really complicated. Then I decided to cut it down to the minimum-viable-product and later add features that my customers need. I spent a little too much time building the prototype. Looking back on it, I could have spent more time on this announcement page.

</comment></comment><comment><USER17> Cool, way to go. Is this much different from the free open source tool s3cmd? <http://s3tools.org/s3cmd>

Tell you what I _do_ have a burning need for: automatic backup of my free, CPanel-based shell account. This includes my databases, my crons, my home directory... I want to be able to jump webhosts in a jiffy if necessary. I would definitely pay for that.

I would also just be comforted in knowing that ALL my data is backed up off site! I'm sure other people have similar concerns. I'd be delighted to pay $5-$10/month for such a service and I wouldn't use hardly any download bandwith so your variable costs would be low.

Consider it.. please?

<comment><AUTHOR> s3cmd is a command-line tool for working with S3. You can use curl, wget, and other standard HTTP clients to work with RestBackup.

I would love to offer a tool that can automatically back up CPanel to RestBackup. Ideally, CPanel Inc. would add this feature into CPanel. Can you run a cron job in your shell account to do the backups? Please join the alpha- test and I can help you set this up. Submit the feedback form to request your invitation: <http://www.restbackup.com/#feedback></comment></comment>