How to set value to login form's submit via themename_form_alter
Buttons and Submits (yes, they are different things in Form API) does not have default value. All you can set for submit is: #access, #after_build, #ajax, #attributes, #button_type (default: 'submit'), #disabled, #element_validate, #executes_submit_callback (default: TRUE), #limit_validation_errors, #name (default: 'op'), #parents, #post_render, #prefix, #pre_render, #process, #submit, #states, #suffix, #theme, #theme_wrappers, #tree, #type, #validate, #value, #weight

As you can see, no #default_value. As user is not supposed to be able to change that value, it's simply #value - no need for defaults.

Read more on Form API Reference.