The reason why you say yes and no is that everybody wishes Christmas is the ideal  and perfect and everybody gets together  but there are so many people who are not in that position, and therefore, it emphasizes those who are not.