Determining the orbitals that overlap to form a sigma bond
The basic answer is go by what is available.

Hydrogen does not have any other occupied orbitals than s-orbitals. Hence it must use an s-orbital for bonding. This goes both directions.

Carbon requires four bonds. In order to achieve this, it must hybridise in some way, otherwise, an electron pair would be in an s-orbital and an empty p-orbital would be unavailable for bonding. Hence carbon uses hybrid orbitals in both directions.

Chlorine could hybridise, but it doesn’t have to fulfill its bonding needs. In general, an electron pair in an s-orbital is more stable than one in a p-orbital because it is ‘closer to the nucleus’. (That layman’s argument is bad but gets us a long way.) So chlorine will happily use its p-orbitals for bonding.
This goes both ways, by the way, so a carbon-chlorine bond ($\ce{C-Cl}$) is a hybrid-p sigma bond.

Finally, let’s consider oxygen and nitrogen. They would love to just bond with their p-orbitals, but even if hydrogen is the bonding partner that leads to too much steric strain. Therefore, they hybridise just enough to alleviate the steric stress while still maintaining a maximum s-character for a/the lone pair. So those would love to form bonds with their p-orbitals but typically can’t (so they must use hybrid orbitals). Phosphorus and sulfur are large enough to generally allow bonding only with p-orbitals.