What are the drawbacks of using an algorithm that is not backwards stable?
Sivan Toledo responded to my query with the following:


  In general, the significance and the broad-context meaning of
  mathematical results is a matter of opinion and debate, so perhaps not
  every researcher would agree to our explanation below. 
  
  In most of the literature, backward stability is a technical and/or
  analytic tool that makes it relatively easy to prove that an algorithm
  produces accurate results (or to bound the inaccuracy). In these
  contexts, the user of the numerical method worries about accuracy, not
  about backward stability, and the numerical analyst uses backward
  stability just as a mechanism to prove the bound that the user cares
  about.
  
  (The analysis in our paper proves accuracy without relying directly on
  backward stability, but this is atypical.)
  
  When an algorithm is backward stable, the user gets not only a bound
  on the magnitude of the error, but also an estimate of its direction;
  it lies in the direction of right singular vectors that correspond to
  small singular values (for the case of solving linear equations). In
  many cases this is completely meaningless to the user.
  
  Backward stability may be meaningful to the end user if the parameters
  of the problem (say the coefficients of a linear equation) are only
  known approximately, say because they are measured or estimated using
  some limited-accuracy physical instrument. In some of these cases it
  may be good for the user to know that the output she/he got is an
  exact solution to a nearby problem.


I'd still like to better understand theoretically what the drawbacks of using inv(A)*b are when b is "bad". That is, if the solution using inv is as accurate as with a "better" algorithm, what really matters. (This is hinted at computationally in my question: the projection matrix A * inv(A' * A) * A' is at least unreliable.)