Are sp, sp2, and sp3 hybridisation only relevant to the carbon atom?
Not only $\ce{C}$,but also other elements (e.g., $\ce{B}$, $\ce{N}$, $\ce{O}$, etc.), which have partially filled $\ce {p}$ orbital can undergo $\ce {sp^3}$, $\ce {sp^2}$, or $\ce {sp}$ hybridization.

To understand how it works for molecules other than carbon, which do not show four bonds, let's take a water molecule as an example. Oxygen atom has $\ce {6 e-}$s in its outer most energy level (n = 2), which has electron configuration: $\ce {2s^2 2p^4}$. When water molecule is formed, it was quantum mechanically argued that all these 4 orbital ($\ce{2s, 2p_x, 2p_y, and 2p_z}$) undergo hybridization to form four equally energized $\ce {sp^3}$ orbitals (the name, $\ce {sp^3}$ is given to the newly formed hybridized orbitals because they were made up by $\ce {1\times s + 3\times p}$). Since they are equally energized, they would occupy the space in such a manner that they have equal distances and equal angles to each other. The only way it can arrange to achieve that is in tetrahedral shape. That four orbital can have maximum of eight electrons. When you put six electrons, which are belong to oxygen in its four $\ce {sp^3}$ hybridized orbitals, you end up with two fully filled and two half-filled orbitals, because you are supposed to avoid pairing them whenever possible to avoid pairing energy so that you keep the atom in its lowest possible energy state. Now, two hydrogen atoms can occupy these two half-filled orbitals with one  $\ce {s}$ electron of each, making 2$\sigma$ bonds. We called it $\sigma$ bonds because there are two nuclei involved to make the bond and it was head-to-head overlapping (precisely, $\ce {sp^3-s}$ bond). What about other two $\ce {sp^3}$ orbitals, which are already filled with electrons from oxygen? We called they are lone pairs. 

Now $\ce {O}$ of $\ce {H2O}$ molecule is $\ce {sp^3}$ hybridized. Therefore, the theoretical shape of $\ce {H2O}$ molecule is tetrahedral. Yet, actual shape is angular because if you get a snap shot of $\ce {H2O}$ molecule, you can't see the electrons. Only you see is three nuclei, one from oxygen and other two from hydrogen, arranged in angular manner.

Similar to water molecule, oxygen atom in tert-butanol (2-methyl-2-propanol), in addtion to all four carbon atoms, is $\ce {sp^3}$ hybridized. The $\ce {O-H}$ bond is $\ce {sp^3-s}$ bond while the $\ce {O-C}$ bond is a $\ce {sp^3-sp^3}$ bond.

Now, you can look how it work for $\ce {sp^2}$ and $\ce {sp}$ hybridization: Only three $\ce {sp^2}$ hybridized orbitals can be made since only three orbitals used to make them. Only way three can arrange in space with equal energy is with trigonal planar geometry (say they are in $xy$-plane). Thus, the $\ce {p}$ orbital not used for the hybridization occupy as $z$-axis to that $xy$-plane. For example, oxygen and middle carbon it attached to in acetone are $\ce {sp^2}$ hybridized so that the $\ce {O-C}$ bond is a $\ce {sp^2-sp^2}$ bond. Similarly, one $\ce {O-C}$ bond in acetic acid is $\ce {sp^2-sp^2}$ bond while the other $\ce {O-C}$ bond is $\ce {sp^3-sp^2}$ bond (both oxygen attached to $\ce {sp^2}$ hybridized carbon).