Asking "which language do you speak?" on the phone
According to Wikipedia:Chinese language


  Chinese Listeni/tʃaɪˈniːz/ (汉语 / 漢語; Hànyǔ or 中文; Zhōngwén) is a group of related but in many cases mutually unintelligible language varieties, forming a branch of the Sino-Tibetan language family


So technically, Chinese is not one language but a group of them and asking "which language, Cantonese or Mandarin?" is correct. 

But when people say "I speak Chinese", it really means "I can speak one of the languages of China" -- it could be Mandarin, Cantonese, Shanghainese or other dialects.

So maybe a better way to ask is: "Do you speak any Chinese language?" And after they confirms, you clarify by asking: 国语吗？ or 普通话吗？ ("Is it Mandarin?") Most would say yes even if they speak more dialects.

I do not think asking where they are from or what dialect they speak is a good idea. Because you would get so many different answers that add more confusion.