Confused on the difference between the frequency spectrum of an entire song, and the frequency spectrum of a point in time
As far as my logic goes, I wont ge able to tell any of musical features just of an amplitude value in the time domain. 


That's wrong. The amplitude-over-time actually is the way the song is played by your sound card (we call that PCM). So, this is your song.

What you really need to understand that your analog audio (i.e. the variations in air pressure over time) is being sampled and stored digitally, and reconstruction happens based on these samples.

Aside from that, you're absolutely right, the Fourier Transform and the time domain signal are equivalent representations of the same signal.

Regarding the question what a graphical representation will tell you: That's quite impossible to answer, as it depends on you ;) Now, there's a few basic things that one can note: For example, if you look at a plot of the time domain signal, and you see high amplitudes in a certain range of time, you can assume that it's loud during that time. Same goes for the absolute of the Fourier Transform: If the amplitude spectrum is large for a range of frequencies, then you can assume that a loud tone of that frequency range is audible.