exception: hosts cannot switch between localhost and hostname
You need to basically reset the config, which is stored in the local database, and then start from scratch.  Assuming you still just have the one server configured as above, restart the instance without the --replSet option, then drop the local.system.replset collection.  Once that is done you may restart again with --replSet and you can re-run rs.initiate() but this time pass in the configuration document instead of letting it auto-discover, something like this:

rs.initiate({"_id" : "my-rs0", "version" : 1, "members" : [{"_id" : 0, "host" : "my-server04:27017"}]})


You can also solve this by making sure that the name resolves correctly and so is automatically populated by the default rs.initiate() command, but the above removes any doubts.