I noticed an error in a graded exam during office hours. Should I give the student the lower grade?
Let the grade stand, but point out your error.

The principle I try to follow is that students shouldn't be negatively affected by faculty mistakes (or that such effects should be minimized).  

I wouldn't suggest adding an additional assignment to maintain the grade, since this is effectively another kind of penalty.  But I would suggest being transparent with the student, explaining your mistake, and showing them where they went wrong (otherwise they are deprived of the feedback that the correct grade would have provided).

If the exam were graded on a curve and re-grading this one question on Student A's test would materially improve other students' grades, I would give other students the boost (i.e. the grade they would have received had Student A's exam been correctly marked the first time) but leave Student A's grade intact.