Correlation as measure of similarity
As you correctly noted, similarity is not a rigorously defined mathematical term. However, "distance" can be defined mathematically. Quoting Wikipedia: 


  In statistics and related fields, a similarity measure or similarity
  function is a real-valued function that quantifies the similarity
  between two objects. Although no single definition of a similarity
  measure exists, usually such measures are in some sense the inverse of
  distance metrics...


Here's a hand-wavy explanation, the math isn't totally precise, but hopefully it's enough to get some intuition:

Let's say we have real valued signals $x(t)$ and $y(t)$ defined on the unit interval $t \in [0,1]$. We will say that these signals are similar if the distance $||x-y||$ is small. Note that there are various choices of how to measure distance. A commonly used distance measure is the squared difference a.k.a. L2 distance defined as 
$$
||x-y||^2_{L^2} \triangleq \int_0^1 (x(t)-y(t))^2 dt \ .
$$

Let's say we have three signals $x$, $y$ and $z$. Normalize all signal energies to 1 so that $\int_0^1 (x(t))^2 dt = 1,$ etc. Suppose $x$ is more similar to $y$ than $z$. This means 

$$||x-y||_{L^2} &lt; ||x-z||_{L^2}$$

or, 

$$\int_0^1 (x(t)-y(t))^2 dt &lt; \int_0^1 (x(t)-z(t))^2 dt$$

or, rearranging terms,

$$\int_0^1 x(t) y(t) dt &gt; \int_0^1 x(t) z(t) dt \ .$$

In other words, the correlation between $x$ and $y$ is higher than that between $x$ and $z$. This shows that in order to find a "more similar" signal, maximizing the correlation is indeed the right thing to do (assuming L2 distance).