Can someone spot anything wrong with my LSTM forex model?
What you are up against is this fundamental property of most tradable, liquid financial price series, and that is, they are Brownian Motion. In discrete time, it's also known as random walk.

The most important property of Brownian Motion is that it's memoryless, whose mathematical expression is

$E[p_{t+1} | p_{-{\infty} : t}] = p_t$

Recurrent neural net, particularly the LSTM flavor, is very powerful in capturing and modeling a long memory process. In fact, it was invented to deal with state that depends on itself many time steps ago (that famous LSTM paper was dated 20 yrs ago[1]). 

What you've demonstrated using RNN (assuming your code and training are free of bug) is precisely this property. Put it another way, there is no way to beat a fair coin in predicting tomorrow's FX price.

Another perspective on your attempt. If such naive model could accurately predict FX time series, it would have been exploited by the whales in the hedge fund industry long ago and the opportunity would cease to exist.

[1] Hochreiter, S., &amp; Schmidhuber, J. (1997). Long short-term memory. Neural computation, 9(8), 1735-1780.