What is the difference between computational neuroscience, theoretical neuroscience, and neuroinformatics (if there is one)? In particular, theoretical and computational neuroscience seem to be synonymous with each other. Neuroinformatics at least seems to deal somewhat more with solving things numerically and the usage and creation of algorithms and programs.
There is no difference between "computational neuroscience" and "theoretical neuroscience" in practice.  The two are almost always used interchangeably.

Neuroinformatics, like bioinformatics, is more about managing data and designing analysis software (that's always somehow integrated with data storage and management).  Generally, it is informational specialists that work in neuroinformatics, as it is basically applied information science (applied to neuroscience, specifically).  In some ways, neuroinformatics can serve as a bridge between experimentalist and theoretician.  Experimentalists use neuroinformatic software to analyze and save their data and store their results... then the theoreticians can take that and make a model, then informatics can design a new program that incorporates the model to help experimentalists analyze their data.  This is a very ideal simplification, but it gives an idea of how the three play off each other.