How inflation affects exchange rates?
Consider two standard relations that tend to hold:

$$i_t = r_t + \pi^e_{t+1} \tag{1}$$

where $i_t$ is domestic nominal interest rate, $r_t$ is domestic real interest rate, and $\pi^e_{t+1}$ is expected domestic inflation. The expression is known as the Fisher equation.  

Also, we have the uncovered interest rate parity

$$ i_t  = i^*_t - [s^e_{t+1} - s_t] \tag{2}$$

where $i^*_t$ is "foreign" nominal interest rate, and $[s^e_{t+1} - s_t]$ is expected appreciation of domestic currency in percentage terms ($s$ is the natural logarithm of "foreign currency per unit of domestic currency" exchange rate). 
Combining the two and re-arranging we have

$$[s^e_{t+1} - s_t] = i^*_t - r_t - \pi^e_{t+1} \tag{3}$$

Assume that inflation is monetary, and keep  $i^*_t,r_t$ constant. Then expectations for domestic inflation will tend to reduce expected appreciation of domestic currency, and may even turn it into an expected depreciation, especially if expected domestic inflation is high. Shortening the length of the time periods, this tendency becomes more of "actual" inflation and "actual" exchange rate.  

The fundamental reason is of course what the OP was thinking: if we think of USD as another good (which it is since it is a store of value and can be exchanged for goods), then, if English pound loses value due to inflation and can buy less goods, why should it be still able to buy the same amount of the good "USD"?