Is it good to mention in a recommendation letter that a student always attends office hours? What kind of impression does this make?
Well, you could really put that like you want to. Attending all office hours could mean a lot of things like the student is extremly insecure and needs confirmation on everything (even though he might be able to do it on his own) or he's not afraid to admit he doesn't know something and asks before doing something wrong. 

It really depends on what you think of the student and if you think he is good enough. From your post we really don't know if attending the office hours was a positive or negative thing. But I wouldn't really write about the office hours themself but more general about the action of the student.