Why might the admin panel's font 404?
The request for the craft font needs to be rewritten, since the actual file is (usually) outside of the public web root. Usually, this will be handled by the location block in your nginx configuration that rewrites requests to index.php. After you added the location block that adds the CORS header, though, all requests for font files are processed by this block only (see this thread for more on location priority). nginx only processes one location block pr request, it doesn't "add them up". The best solution would probably be to make the location block for adding the CORS headers more specific, so that the regexp doesn't match the Craft fonts.

Here's an example on what I did on a previous project where the fonts were inside my /assets folder:

location ~* ^/assets/.*\.(?:eot|ttf|woff)$ {
   add_header Access-Control-Allow-Origin *;
}