How to disable core JS aggregation and AdvAgg just for admin paths
So, the issue I describe in my question is this Drupal Core issue: Drupal.ajax does not guarantee that "add new JS file to page" commands have finished before calling said JS

So, my solution wasn't to disable the aggregation, but to apply the latest patch provided in the issue on Drupal.org