Element API: return a related entry's asset data with entry data
Sounds like you need to do some eager loading of your assets.

Here are some examples of how to do that in the Element API plugin:

ElementAPI handling CategoryModel and returning a heirachical object

Getting categories in elementAPI performance issues and help required