How to prevent too many students from making appointments to meet with you when you put "Office Hours: ..., or by appointment" on the syllabus?
What an interesting problem to have. When I saw the title, I thought, "This must be hypothetical, because this problem never, ever occurs." But I was wrong.

Anyway, I don't know how to deal with having too many students try to schedule office hours, because I have never had that problem, even when teaching 60 students. 

I do, however, employ the message board tactic that you mentioned, which probably helps cut down on office hours (and dramatically reduces the number of emails I get). I suggest the free (and extremely slick) service Piazza. Students can collaboratively (wiki-style) answer each others' questions, and instructors can edit everything in sight, as well as create their own answers. When posting questions, students can also appear anonymous to their classmates, but not to the instructor, which eliminates any embarrassment they might feel. 

tl;dr: Try Piazza.