Can't change my store back after adding prepaid credit card
you probably have money left on your Apple account which you created in Japan. the other possibility is that you may have some subscriptions to which you subscribed in Japan (for example, Apple Music, any news papers and so on). 

so you would have to spend all that money or unsubscribe to be able to change the store