Most robust way to stiffen a thin PCB
Without having a much clearer picture of why you need to stuff the end of a circuit board into the AA battery holder it is hard to make very good suggestions. 

If I were you I would rethink things a bit. When you make testing gadgets one of the most important things to consider is that the parts of the gadget that plug and unplug or probe into something will wear out and break after a certain amount of usage. This means that you should strive to find simple low cost things that can be replaced easily to serve as the connection between the DUT (device under test) and the testing gadget. That will typically mean that there is a connector of some sort at the testing gadget end of the interface. 

So taking all that into account I would use a length of fairly heavy gauge wire for the probe connection which connects to your test setup via a screw terminal block, push on crimp connector etc. In the other end of the wire crimp on one of these style terminals which will easily push into the AA battery pack: