Safari does not restore saved tabs when I exit via X icon
Closing the window is not the same as quitting the app. 

Clicking the red X closes the window, telling Safari that you've done with the contents - leaving nothing for it to remember. 

It's not clear from your question quite what you are selecting from the right click menu, but anything other than quit - or quitting after all windows are already closed, will do the same. 

Note that, if you do forget &amp; close the windows before quitting, that you can use History menu > restore all windows from last session to get them back again.