There is data for the field - On uninstalling module
This is a known limitation to Drupals Core and this finally gets resolved for Drupal 8.5.x, see this issue:
https://www.drupal.org/project/drupal/issues/2282119

But for versions earlier there is an option to workaround this, by creating a form that handles the cleaning of added data to make the uninstall validation pass. This is a UI solution but you still need to add a not somewhere that users know how to find the form. See how the Flag module handled it:
https://www.drupal.org/node/2409673

There are other options like creating a drush command, but what you basically need to do is to clean the added data from the database to make the validation for uninstall pass and allow users to uninstall the module.