After any AJAX form event, all Entity Reference Fields are blank
I finally solved this.

There is a patch here... https://www.drupal.org/files/issues/entityreference-use-tokens-2010898-126.patch That allows you to use tokens in the Entity Reference Fields "Pass argument to view" field.

The problem was that after AJAX, the view I was using for the Entity Reference was losing its contextual filter argument. This patch, and the use of the token fixes the problem nicely.