What are the mathematical prerequisites to be able to study artificial general intelligence?
I always recommend starting with game theory, combinatorial game theory, and algorithmic combinatorial game theory, (but I'm potentially biased;)

Combinatorics is a given--discrete mathematics is heavily utilized in computer science, and, with the advent of Combinatorial Game Theory (CGT), ability to determine if a given choice can be deemed optimal ("perfect play").  CGT arises out of traditional Game Theory, which we sometimes term "economic game theory" to make the distinction.  Out of Game Theory also arises subfields such as Evolutionary Game Theory, which is important in AI.

These fields relate to rationality, which is the basis for optimized decision making. Decision making algorithms seems to be the fundamental distinction of what constitutes an Artificial Intelligence. 

From minimax to gametrees, it's probably a good idea to have a basic grounding in these fields, even if the problem you're AI is trying to solve isn't formally defined as a game.  

All problems, from a fundamental standpoint, can be regarded either as puzzles--non-competitive context--or games--competitive context.  This distinction is based on whether there is a single agent (puzzles) or multiple agents (games.)