What is expected in a masters thesis of a mathematics student?
I think this varies a lot. But for Germany your first question can be answers succinctly: In a Master's thesis you should show that you have potential for research.

On the other hand, expectations vary a lot between advisors. But certainly you do not have to prove a new theorem or develop a new theory. 


  
  How much is it valued (if at all) when one applies for PhD?
  


I can only answer for the situation where you apply in Germany. The thesis can be a door opener if it is topic closely related to the field where you want to do a PhD. Also a good mark is important. But also in Germany hiring professors will often contact your advisors or request a reference letter and this is much more important.


  
  I have heard that its value is more in Europe than America, which if I were to guess I would say, may be due to no GRE like criterion. Is this true?
  


Not sure on this point since I can't provide a comparison with the US and also I am not sure if the situation is uniform with the EU.