Prevent form rebuild on image upload
I've done some tests using an Entity Reference field, and setting its default value in a hook_form_alter implementation. Everything is working fine - when I upload a single or multiple image files, the default value that I set in the hook for the Entity Reference field is preserved, and is saved fine when I press Save.

Please notice the way I'm setting the default value in my implementation of hook_form_alter(). This is different from what you have:

$form['field_reference_to_page'][LANGUAGE_NONE]['#default_value'] = 'trip (189)';

(My field is called field_reference_to_page. In your case, you 'll need to replace with field_boardgame). Notice that the arrays are different from what you have. Basically there is no [0] or ['target_id']. The value 'trip (189) is the title of the node and the node ID that this field is referencing. I made sure that the node really existed, and the node ID was correct.

I put together a short video proving that it really works. 

By the way, Image fields cause an AJAX call when an image is uploaded - the form is rebuilt but not reloaded. But as I said, the default_value is preserved if you implemented the form alter correctly.