what is the general name of this problem? what is the general name of a problems where learning agent observe new data as a learning goes on. For example when playing platformer games one must incrementally learn new level areas and states while remembering all previous states so as to successfully proceed through the level.
The machine learning subfield you are asking about is called Data Stream Mining.

(However I'm not sure about its relationship to platform games)