CTools modal return incorrectly return html instead of ajax commands
Turns out the offending part is not the logic, but the function name. The my_module_user_login_form function happens to be a invoke hook form, which interferes with the modal.

Changing the function name solved the problems.