How can a changing signal be Fourier transformed? Fourier transformation is used to split a periodic signal into frequencies, and calculate the phase and shift for each frequency. But what if we would like to record an audio, which is changing, for example, because someone starts to play with a new instrument?

I know that if the recording is 44100 Hz, then we record 44100 samples in each second.  But how is it Fourier transformed? The only way I can think of is that one second is divided into 25 segments, and we do the transformation to get the phase and shift for each frequency in each segment.
You are correct that it is often broken up into chunks. There are multiple ways to combine those chunks (Google: fft overlap save).

It is often important to window the data (again Google: hamming window fft). One critical reason to do this that gets to the heart of your question: the fft assumes the chunk you fed it continues infinitely forward and backward in time. Obviously an audio file does not have this property.

For your example you might want to chunk your audio file into groups of powers of 2 like, 1024,2048, or 4096.