How do I create a file with a custom file extension?
I tried to recreate your Problem and noticed that you probably have not enabled the "Show all File Extensions" option in the Finder Preferences. 

You can check by selecting the Finder and selecting "Preferences" from the Menubar. A small window should appear, titled "Finder Preferences". Select the "Advanced" Icon at the Top of that window. You should now see the option "Show all File Extensions", it should be checked. All files in your finder window should now show their extensions, like .txt or in your case .crt

You can now easily change all files to another extension.

If that does not work then the Fileextension is probably linked to another application. .crt-Files should link to the Keychain-Application which handels all certificates. You can check by selecting the crt-File and pressing "command" + "i". An information window should appear with additional information to that file. At the Entry "open with" should appear "Keychain.app".