If left stationary at a specific height, would altimeter change the displayed value over time?
You are correct in both regards -- as Terry's comment points out, an undisturbed pressure altimeter's reading (imagine one sitting on a bench outside the local hangar) will change with the local barometric pressure.  TSO'ed "sensitive altimeters", i.e. ones with a hundreds pointer, not just a thousands, are required to have a knob and window, called a Kollsman window, for setting the local barometric pressure, or standard pressure (29.92) when above the transition altitude (18,000' in the US, check with the local aviation regulator for where you are in the world).  "Glass cockpit" systems have a knob/function for the same job -- whether it be a dedicated knob, or a mode on a multi-function knob.