How to convert the drupal-entity tag into an img tag?
Basically you'll want to get the field's value and do a str_replace(), but where?

I haven't done this myself yet, but I believe you'll have to create a normalizer.

Serialization API overview (good read).

I think you can achieve it by doing the following Changing the way Serializer handles entities.

There is more indepth info here Using Normalizers to Alter REST JSON Structure in Drupal 8.