Why don't exchanges offer all coins? What is the tech overhead associated with supporting a new coin on an exchange (such as bittrex). Especially when most of the new coins are built on the popular ones (ETH, LTC etc.)

If it's relatively simple, why don't exchanges offer all coins?
Except ETH (even that too have some level of changes) and Waves (and soon NEO and others in future), all the other type of crypto coins require some level of expertise before it can be added in the exchange.

Also, the more crypto coins added, the more possible pairs will be possible or required. Example, let a coin name be ABC, the possible pairs can start with just

ABC/BTC

Then with time and popularity and demand, the following can be required:

ABC/ETH

ABC/USDT

ABC/CNY and so on.

This puts more load on the server than the exchanges get back in fees. Also, some coins may have some technical problem often, which cost the exchange time and expertise to solve.

That is the main reason popular exchanges like Bittrex and others only list a limited ones (mainly due to server load). While less famous exchanges list almost all the crypto coins to get any kind of traffic they can get. If that particular exchange is the only one allowing that coin, then they will get the initial traffic.