What would the altimeter indication be through a cold front?
There are two altimeter reading questions here, what happens when:


flying towards the cold front (third paragraph)
flying into the cold front (headline)


According to this University of Illinois web article following things happen:

Flying towards a cold front (or in the article the front is approaching you, but it makes no difference):


pressure drops steadily
temperature is steady


Passing through the front:


pressure reaches minimum, then rises sharply
temperature drops suddenly.


So, to answer both questions, assuming you are flying at relatively low altitude and maintaining it in reference to your altimeter and you do not adjust QNH:


Approaching the cold front you would steadily drift downwards: from high to low, look out below!
Passing through the front you would need to climb to maintain steady altimeter reading: low to high, hello sky!


You see, the question is about what happens to altimeter reading, but since the vertical position of the aircraft is maintained in reference to the altimeter reading, whether it's you or the autopilot flying, the altimeter reading would not change. The vertical position of the aircraft would be instinctively or algorithmically adjusted to keep the reading at a selected value.

What happens at altitude, is (for now) somewhat beyond me (pressure gradients and stuff), so  I dare not answer that scenario here yet.



P.S. a private pilot should not fly through a cold front: it's not a good place for small planes to be in...