Why only sine waves?
Because sinusoids have some important mathemtical properties. The first being how they behave under differentiation and integration.

$$\frac{d}{dt}\sin(\omega t+\varphi) = \omega\cos(\omega t+\varphi) = \omega\sin(\omega t+\varphi+\frac{\pi}{2})$$

In other words when we differentiate or integrate a sinusoid we get a sinusoid of the same frequency. The sinusoids are the only periodic functions (from the reals to the reals)* for which this is true.

The second being how they behave under addition. Two sinusoids of the same frequency but different phase add together to make a sinusoid of the same frequency (unless they are equal and opposite in which case they cancel to produce zero).

$$a\sin(\omega t)+b\sin(\omega t+\theta)= \sqrt{a^2 + b^2 + 2ab\cos \theta} \sin(\omega t+\operatorname{atan2} \left( b\,\sin\theta, a + b\cos\theta \right))$$

These properties mean that when we feed a sinusoid into a linear time invariant system we get a sinusoid of the same frequency out. Many real-world systems behave to a first approximation as linear time invariant systems, especially for small signals. We can characterise a linear time invariant system by measuring its magnitude and phase response to a sinusoidal sweep and then we can predict its response to other signals by breaking those signals down into combinations of sine waves and then applying the superposition principle.

If we tried to do a similar frequency sweep test with any other waveform we would have an output waveform a different shape to our input waveform, which we would have to deal with somehow, making the characterisation process much trickier.



* As has been pointed out in the comments the exponential is it's own derivative, but the exponential of a real variable is not periodic. The exponential of a real variable multiplied by the imaginary unit is periodic but produces a complex result. If we decompose it into it's real and imaginary parts using Euler's formula then we are back to a pair of sinusoids.