Solving second-order ODEs with Taylor series approximation of second derivative and linear algebra I was told about an algorithm to solve ODEs which I thought was very clever. I am sure its not something new. It works by discarding the $\mathcal{O}(h^2)$-term from the usual formula for computing the second derivative:

$$f''_i = \frac{f_{i-1} + f_{i+1} - 2f_{i}}{h^2} + \mathcal{O}(h^2), \qquad i = 0,1,2,\cdots,N$$

one can rewrite the equations for the derivatives for all $i$, as a set of linear equations, then use smart Gauss elimination from linear algebra to find expression for $f$ with exactly $5N$ floating point operations.

(1) Does this method already have a name?

(2) Should I be excited about this algorithm? Is this a particularly good method? Or are there other very different algorithms for solving ODEs which are much better?
What you have there is the central difference approximation to the second derivative. The Wikipedia article on finite differences explains this pretty well.

Of course, this isn't a method as such; that depends on the exact problem being solved. If this were an initial value problem, then a simple rearrangement of this equation is typically known as Störmer–Verlet integration (or just Verlet integration depending on whom you ask).

If it was a bounding-value problem, then this discretisation of the derivative gives you a tridiagonal system, and you don't have a complete "method" until you have a method to solve it. A small modification of Gaussian elimination is quite efficient on matrices of this form.

As to competitors... that's a very long discussion. Any introductory textbook on numerical analysis should give you a good start.