How to hook into core's "log in to comment" link
I found the solution with the help of this SE post. Using MYMODULE_node_view_alter(&amp;$build) and MYMODULE_comment_view_alter(&amp;$build) in my custom module, I was able to change the values using: 

$build['links']['comment']['#links']['comment_forbidden'] = str_replace('%23','#', $build['links']['comment']['#links']['comment_forbidden']);