How to generalise the Fourier transform? The Fourier transform takes a signal and splits it into a series of sine and cosine waves.

I am told that it's supposed to be possible to split a signal into some other set of functions. My question is: How do you do this?

I'm presuming that the set of functions you use would have to have certain properties for this to work. (E.g., you have to have "enough" different functions to capture all the information of the original signal.) How do you figure out whether your set of functions is suitable? And then how do you do the actual splitting?
The Fourier Transform is just one of so many different transforms that alter the representation of (usually) a time-series from the time domain, to another domain (usually a frequency domain but other representations exist for other transforms such as time / frequency, time / scale and others).

You can find much more information about transforms in general from this Wikipedia list of articles that lists some popular and often used transforms. (You might want to focus on the Discrete and Integral Transforms at first)

Alternatively, you can check out this recent discussion on how the Wavelet transform, achieves a decomposition similar to that of the Fourier transform.

Finally, when you have the luxury to have acquired simultaneously many different time series from the same phenomenon you can even employ techniques such as Principal Component Analysis (PCA) and Independent Component Analysis (ICA) which go to the point of transforming a signal to a sum of elementary waveforms that are actually extracted from the signal itself (rather than being pre-set as it is done in the Fourier (and related transforms) or Wavelets).