How to implement AI strategy for Mastermind
You could possibly apply neural networks, reinforcement learning to summarise results of previous choices (what you are calling context) and use score predictions to suggest the next turn's guess. However, the game of Mastermind has a small search space and it is possible to process this "context" more directly by refining a set of guesses. This will be much more efficient and simpler to understand than a neural network approach. It would be very hard to make a neural network variant which was as efficient - either in terms of CPU time, or in terms of number of turns it takes to find a solution.

In practice, a Mastermind solver is much like a Hangman solver, or a Guess Who? solver. You have an initial large set of all possible answers, and need to narrow it down to a single correct answer. You do this by processing after each guess to reduce the set of answers that meet all the constraints that the game has given you so far.

The agent needs to know the score function that compares a target value with the guess and returns the score. Let's call that score(guess, target)

The algorithm looks like this:

(Opponent sets unknown_target)
Initialise possible_answers as list of all valid targets

For each turn:
  Select one of possible_answers as this turn's guess
  Ask opponent for gscore = score(guess, unknown_target)
  If gscore is [0,4] then exit(win)
  If it was last guess then exit(lose)
  For each possible_answer in possible_answers:
    pscore = score(guess, possible_answer)
    If pscore != gscore, then remove possible_answer from possible_answers


You can finesse this for the stage Select one of possible_answers as this turns guess by trying to optimise either by psychological model of opponent or trying to find choices that are likely to cause the best reduction in size of possible_answers. However, a simple random choice should do quite well.

Also worth noting is that the algorithm does not depend on the exact nature of the scoring function, so it is applicable for many variations of guessing games. It does rely on the score providing information that will reduce the remaining set of guesses. In some games that may mean taking more care about the precise nature of a guess, in order to maximise this effect.

Out of interest, I implemented this algorithm and tested it when there were 10 choices at each position (i.e. digits 0 to 9), maximum of 10 guesses allowed, and the target to guess was set randomly. Using random guesses and the algorithm exactly as written, the above approach guessed correctly 9,996 times out of 10,000, and on average the guesser won the game in 6.2 turns.