Can I play 4k video on my TV through my MacBook using a Thunderbolt/HDMI cable?
Yes, the correct mini-DP to HDMI cable will work and your MacBook is not limiting this.
Have a look at the tech specs from your MacBook Air or read this article on macrumors.