What units do I use for energy in the Boltzmann distribution?
Any quantity that is used as a power has to be dimensionless (that is, a pure number with no units). This means that $kT$ has to have the same units as $E_i-E_j$, so the terms in the exponent cancel. If you're measuring temperature in $\pu{K}$ and Boltzmann's constant in $\pu{J/K}$, then the units of energy you have to use are joules, $\pu{J}$.