Drupal Forms API how to get degree sign onto submit button
Buttons in Drupal by default are being generated by the theme_button() theme function which uses drupal_attributes() to handle all of the button's attributes which always escapes an attribute's value using check_plain().

Visit api.drupal.org/api/drupal/includes%21form.inc/function/theme_button/7 for the theme_button() function and click under the Code section on a function name to see what it does.

I think there are two possibilities:


You write your own implementation of theme_button(): copy the original function, paste it into your template.php file and rename it to something like YOUR_THEME_button();
Or just use the bare unicode character as you did in your question. I read at stackoverflow.com/q/3627720/328272 that this does not work on Windows XP, but I successfully tested it on IE7 and IE8 on XP using browserstack.com.