Explaining temperature change in an exothermic system.
In thermodynamics, we like to divide any situation into two parts, the system and the surroundings. In general the system is the part of the universe that is being studied while the surroundings is everything else that interacts with the system. Between the boundary of the system and the surroundings, energy and heat can be transferred (in an open system). 

So in an neutralisation reaction, the system is the actual acid and base while the surroundings is the solvent (the water). Since it is an exothermic reaction, this means that energy is released from the system to the surroundings. Since the water gains more kinetic energy from the system as the reaction occurs, its temperature will increase. Also this means that the system loses energy and therefore become colder. So to answer your question, the solvent gets hotter while the system gets colder.