What does it mean to be on a short list?
Nope, there's no widely agreed upon definition of "short list".  Sometimes a committee draws up an official short list, while sometimes people informally talk about a short list based on who is being seriously considered even if there's no official list.  Sometimes the short list consists of the people being interviewed, sometimes it's the people for whom letters are being requested, and sometimes it's in between.  Things can get more complicated when there are several forms of interviews (e.g., interviews at an annual meeting vs. fly-ins).  Sometimes a reference to the short list is a meaningless pleasantry meant to soften the blow of rejection when a committee member knows an applicant personally.  Ultimately, the only thing you can say for sure is that it's better to be on the short list than off it.  Beyond that, you have to guess from context or ask for clarification.