Find offset that minimizes squared difference between two vectors of different size
Without changing the algorithm, the only things I can see by having a look at the assembly of sqDiffSum:


read x[i] only once in the outer loop (i.e const double x_i = x[i] and then use x_i instead). You'll get a small win by not having to fetch it whenever you enter the if.
compute once i * scale - offset in the outer loop because this invariant. But latest gcc and clang seem smart enough to figure this out themselves.


Also, you might want to test std::unordered_map.

edit: whenever bin &gt;= ref.size(), you know that it will be for all the next j. Maybe you even replace j by bin running (before morning coffee) when \$offset &gt;= i \cdot scale\$ and \$\frac{i \cdot ref\_size}{x\_size} + offset &lt; ref\_size\$