Food for comprehensive or qualifying exam?
My question is, how often do students do this for qualifying or comprehensive exams?


At my previous department (in ECE, if it matters), students do for qualifiers/prelims what they do for their defense, the only difference being that you scale down the total quantity of refreshments that you provide/purchase a bit if the exam is not open to the public.


  Would committee members expect at least some coffee?


I don't know about "expect," but it's nice to have some on hand: at all of my exams, all of my committee members went straight for the coffee upon arrival to the exam. 

Edited to add — The oral portion of the qualifying exam at my previous department consisted of a presentation which was meant to showcase any research highlights the student had achieved up to that point and future directions. If your qualifying/comprehensive exam is like that described by @CameronWilliams below in the comments, then bringing refreshments may not make any sense, i.e. your mileage may vary.