Should sink overflow be full of water?
Is that normal?

NO, It should not be full of water. It's purpose is to drain water that flows into it so water does not spill over the top of the sink and on the floor.
There is channel that the overflow water flows through to an opening in the drain. (If the proper drain with an opening was installed, some are not designed for an overflow).  If the installer used plumbers putty it is possible that the openings got clogged.
If you take the stopper out you should be able see the opening on the side of drain hole and possibly see if the hole is obstructed.