How can I incorporate domain knowledge to choose actions in the case of large action spaces in multi-armed bandits?
You will want to look into Contextual Multi-Armed Bandits. These are MAB problems that additionally involve feature vectors in some way. 

You'll sometimes see researchers considering problems where you get to see a single feature vector per timestep (like an "environment state" you're in) which may provide useful information. You'll also sometimes see researchers considering problems where you observe a single feature vector per arm/action (per timestep). That second case is pretty much what you're describing; every arm/action has a feature vector (model, year, mileage of car), and you can use those to make predictions and generalize across arms.

To give a little bit of flavour for these algorithms, the most common ones simply assume that the rewards you observe after pulling arms are given by a linear combination of features, plus some noise. Then, they simply try to find (through online learning) a parameter vector such that observed rewards can be accurately approximated by the dot product of the parameter vector and a feature vector. Of course, there are lots of different variants of algorithms, and not all are linear, but this gives an idea of what they generally look like.