OpenAI Gym interface when reward calculation is delayed? (continuous control with considerable reaction time)
It's not on your end, as a creator of flight simulator, to worry about what action should get the credit for the reward that happened some time after the action was taken. You should return the reward when the actual event happens not when the action that caused it happened. It's the job of the reinforcement learning agent to figure that out. For example if you want to give a reward when airplane nose is at 45 degrees from the horizontal axis, you should return the reward when that event actually happens, RL agent should figure out that the crucial action happened some time ago. This may be difficult for the agent but its up to the user to use a proper algorithm and proper exploration strategy to solve the problem.