How do I force a parent-child object hierarchy to follow armature *without* using armature modifier?
I'm assuming that you've already copied the object's starting rotation to match the rotation and roll of the bone. You might need to apply this rotation (Ctrl+A) in order for the constraints to start working properly.

Another possibility is that you might be experiencing Gimbal lock on your objects. Armatures don't exhibit this problem because their default rotation type uses quaternions, whereas objects default to Euler rotations. Fortunately, you can change the rotation type for objects (3D View > Properties region (N) > Transform panel > Rotation) from XYZ Euler to Quaternion (WXYZ).

Try that and see if you're still running into rotation issues.