How do I get a single IK bone to move up when the controller pushes against it?
The IK goal is to manage a chain of bones and bend them from point A to point B with an orientation towards point C. Here you have just one bone, you should use a damped track constraint instead, which has the simple role of pointing a bone towards another and seem to be what you need.
Then, before the damping track, you can add a Child of with only Z translation enabled. Don't forget to hit the set inverse button.