What is drawback of smaller diameter recessed lights?
There really isn't any downside that I'm aware of. 

Bulbs are a concern, although that applies to any type of light: are they readily available? Can you get them from your local big box store, or do you have to resort to a speciality lighting store, though special order, or from some obscure site on the internet?

Regardless of light size, providing the appropriate number of lights, in the appropriate positions, and with the appropriate bulb beam angle (using flood vs spot) is important. I actually discussed that in depth on my answer here: Is there a good rule of thumb for how many lights you need in a room?