Caulking a corner instead of taping with joint compound?
First I would not even have a conversation with that contractor.   Gone instantly.   If he is cutting corners here, what else would he do without you knowing?   Bye.

Caulk in corners - I have had to do this as a temporary fix but here are your issues:


your corner isn't really a corner.   Corners look good because of the sharp angle.   You can for sure see if a corner has been caulked and it looks off to the eye.   If you don't care about aesthetics then no big deal.
paint does not adhere to caulk like it does drywall/mud.   No matter if you use the most paintable caulk in the world, the caulked part will look off.   Paint will chip, paint will be more easily discolored, and touching up painted caulk will look lumpy.
caulk will not last unless the home environment is both well conditioned and very stable.   So you might get away with caulking an inside wall on the first story of a two story house with a basement that is always set to 68F.   Might last you 7-10 years if you don't touch it.   Doing a wall on a ranch in a temperate climate, the drywall will shrink and expand, make the best caulk pull or get brittle within a year or two and you will have to constantly have to recaulk.   Simply not a long-term solution.   


But the fact that this topic was breached with a home owner is a huge red flag.   No normal contractor that does drywall would even have this thought in their head.