Do graduate schools care about published solutions of contest problems in Mathematical magazines?
Sure, that would certainly be viewed favorably.  How favorably would depend on the rest of the application and the institution (though I can't think of any situation in which it would play a decisive role), but if the question is having done it whether to list it, then the answer is a clear yes.

If on the other hand you haven't done the problem-solving yet but are wondering whether you should engage in this activity for the value it has in your future grad school application: I don't recommend that, but I can think of plenty worse ways to spend your time.