Problem with text format and base64 inline images
I believe I tracked this down. With the filter formatter you mentioned, it calls Xss::filter() on the text.
Xss::filter() looks at the src attribute and will strip any protocol that have not been allowed. It sees anything XYZ: as a protocol, in this case data:.
I added - data to my docroot/sites/default/services.yml filter_protocols value and stopped the data: from being stripped.
I don't know if this opens any other security concerns or not.