How can I add a bone constraint to an armature bone if it only lets me select the individual bones outside of Object mode?
To create a constraint follow these steps:

1) Select a bone in your edit mode; in this way you are working with your armature and not with your main mesh



2) Then switch from Edit mode to Pose mode



When you clicked on Pose mode you'll see your selected bone changing color from orange to blue

3) Now you can select the bone you prefer and create whatever constraints you need. You found all the types of constraints here in the image below