How to copy the phone number of the sender of a text message?
The message itself reacts to a long press, but the contact bubble on its left reacts to a single tap. Then you can long press on the phone number in the bottom field, and you can copy the phone number from there. 

Edit:

There has been a change in in Android 6.

This answer does not work with the new message application.

With Messengers (as opposed to Hangouts, you can't copy the phone number by tapping the bubble icon.

Instead, tap the three dots in the top right, then find Participants et options (not sure what it is called in english), long press the bubble the label or the bubble of the person you want to copy the phone number from, then copy.

Unfortunately, the actual copy operation behind the menu doesn't seem to be implemented. The phone number is not copied using this operation..

This is inconsistent with the previous way, but it makes more sense in a way, if it worked.