user_login_finalize() logins the user, but I remain anonymous
You mean that the user remains anonymous in the browser session, right? (In Drush, you must pass -u uid on every invocation to specify the user, because Drush is stateless.)

Make sure that the Drupal session cookie is being set in the browser.  If, for example, you set the cookie in the secure context (https) and then redirect to http, the cookie will be inaccessible, and the user will remain anonymous.

Edit: user_login_finalize requires a $form_state parameter; see: http://api.drupal.org/api/drupal/modules%21user%21user.module/function/user_login_submit/7  It's hard to see which fields of the form state are relevant, since it is just passed to user_module_invoke.  However, if you compare what is in $form_state during a normal Drupal login, and supply the same information in your function, then it should work fine.