which layer of rigid foam should i caulk when insulating rim joist?
I would caulk on both of them. The purpose of the caulk it to prevent air leaks and convection, and not sealing both will reduce the effectiveness of the insulation a bit.