Collision not working after moving armature/setting different pose
So I was able to find the answer for my problem: I parented the chest_collision object to the armature with empty weights instead of parenting it to a bone. Moving the armature would move the collision object like I wanted but the collision seems to be calculated based on the object's origin.

Sure enough, going into edit mode reveals it at the location of its origin:

Parenting the chest_collision to one of the neck bones fixed the problem!