What is the Fourier transform of $\delta(t-a)$
Your argumentation is perfectly right. The only thing you are missing is that the sum of complex exponentials becomes a comb function in the limit. See also this link and this matlab code:

N = 2048;
n = (0:(N-1))';

k = 4*(0:20);

signals = zeros(N,length(k));
for kk = 1:length(k)
    signals(:,kk) = exp(-1j*2*pi*k(kk).*n/N);
end

plot(abs(sum(signals, 2)));