Custom Application Keyboard shortcut for Safari not always working
This is a known “bug” which I don’t know if Apple will ever consider a bug. 

The problem is that the list of browsers is generated “dynamically” each time you click on it (or at least the 1st time after Safari has been launched); the reasoning behind this is that different users will have a different set of browsers, so the list cannot be hardcoded. So given the above, the shortcut fails to work until the list has been “generated” by Safari. This occurs when you display the menu. 

A possible solution would be to use Automator and create a service that opens the current safari page onto Chrome. But that’s beyond the scope of this answer.

TLDR version: There’s nothing you can do other than file a bug report at Apple if you want to stick with the same shortcut technique. Alternatives involve Automator.