Reinforcement learning: How to deal with illegal actions?
In my project I also had the problem that the action space is not the same for every state of the environment. I do not like the approach to penalty forbidden actions with a high negative reward since it feels a bit like cheating. However it might work, I just haven't tried it. 

The approach I used, which you could apply as well, is to integrate an additional function into your action space. This function would map an action to a specific amount of kW. Thereby, depending on the current state, the function maps the action to the amount of kW to charge or discharge your battery with. This has the advantage that you do not have to deal with illegal actions.

This could be applied as follows:
Instead of defining for every action the amount to charge / discharge your battery with, you create a set of functions that defines the respective amount. Here an example with five actions:


Action: Discharge the battery entirely
Action. Discharge the battery so that half of its capacity remains,
otherwise do nothing
Action: do nothing
Action: charge the batter to half its capacity, otherwise do nothing
Action: charge the battery to its maximum capacity