JS batch progress bar: possible to force refresh more frequently than every 1500 ms?
Turns out none of us (neither I or nor the commenters) were completely "right", nor completely "wrong". There IS an arbitrary limit hardcoded in D8's core, for JS batch progress bar refreshes, but it's not 1500 ms as I thought -- it's 1000 ms.

And it's harcoded in a different place than any of us thought.

As @MarioSteinitz stated, yes, the batch API inserts only a very short pause between two Ajax requests -- 10 ms. (The Ajax API is different with its 1500 ms default pause and I mixed up the two.)

However, try as I might, I couldn't get the progress bar to refresh more often than once every approximately 1.1 or 1.2 seconds. No matter how fast I finished my simulated batch work (by reducing my usleep() in the batch code), the relevant DOM objects, such as progress__bar, were never refreshing more frequently than just over once a second.

You can easily see their "throbbing" in Inspector's DOM view and count onethousand-one, onethousand-two as they pulse away... even when my batch function did absolutely nothing, the fastest I could get these DOMs to flash was just over once a second.

So what gives?

Well the batch API is just weiiiiirrrrrd... instead of giving the programmer the say in how much work ONE Ajax request is supposed to do, the function _batch_process() in core file /core/includes/batch.inc has a loop harcoded that calls the batch function again and again until at least 1000 ms have passed (or the batch is finished):

// If we are in progressive mode, break processing after 1 second.
if ($batch['progressive'] &amp;&amp; Timer::read('batch_processing') &gt; 1000) {
  // Record elapsed wall clock time.
  $current_set['elapsed'] = round((microtime(TRUE) - $current_set['start']) * 1000, 2);
  break;
}


Weird because a batch is ALREADY supposed to be a large process chopped into small bite-sized chunks. So the entire batch already IS a loop, but there's, in fact, a secondary inner loop, whose break conditions are outside of the coder's control!

You think this kind of policing of the programmer's wishes is not weird, arbitrary and unnecessary? Well consider that that the batch API (after what feels like decades), still doesn't allow progressive batches (i.e. batches that are all processed in one swoop, for example from the shell) -- see https://www.drupal.org/project/drupal/issues/638712.

But I digress.

So for sh*ts and giggles I changed the > 1000 to > 100 in the above core code and what do you know, the progress DOMs refreshed like crazy! 

Of course I can empathize with the idea that no batch should hammer its host server with heavy requests every 10 ms, for obvious reasons. So the developers decided to loop and re-loop the batch function until at least 1000 ms have passed, before the next Ajax call is triggered.

Well-intentioned? Yes! Arbitrary and inflexible? Absolutely. As the programmer of a batch, I believe it should be MY choice how much exactly I want to do in one Ajax call. Any Drupal code has the potential to crash or deadlock the server, or overwhelm the browser. Why police this particular part?

Drupal maintainers: Why not default to 1000 ms if that is a trusty value for most applications or unaware coders, but for batches that require a more dense flow of output, let's say, once every 100 ms, give us the power to do that!

So how to get rid of the pesky > 1000 condition - without hacking the Drupal core of course!? 

if ($batch['progressive'] &amp;&amp; Timer::read('batch_processing') &gt; 1000) {


If this were D7, we'd just manipulate the global $timers array: Just before our batch function exits, set its start value back to the stone age (01/01/1970), and the calling _batch_process() will always "think" at least one second has passed, and give control back to Ajax immediately.

Just that this isn't your grandma's Drupal; we are now all fancy object-oriented and stuff.

Looking at https://api.drupal.org/api/drupal/core%21lib%21Drupal%21Component%21Utility%21Timer.php/8.2.x we notice that we can't just poke whatever we want into the Timer::$timers array any more. It is declared protected, i.e. no outsider, such as my batch function, is allowed to mess with its internal values.

So we're back to square one... or are we?

Well thankfully this is PHP and not a "real" programming language! (ducks)

Meaning, there's always a loophole to bypass any new-fangled OOP restrictions that those eggheads at Software Engineering University demanded. To hell with them!

After some googling, I wrote the following function. And by "wrote" I mean I all but stole it from people who know much more about PHP than I do: https://stackoverflow.com/questions/6448551/is-there-any-way-to-set-a-private-protected-static-property-using-reflection-cla

  /**
   * Revert Timer::$timers['batch_processing'] back to the stone age
   * to trick batch.inc / _batch_process() into thinking at least 1 second has passed
   */
  protected static function revertTime() {

    $obj         = new Timer();
    $refObject   = new ReflectionObject($obj);
    $refProperty = $refObject-&gt;getProperty('timers');
    $refProperty-&gt;setAccessible(true);
    $myTimers = $refProperty-&gt;getValue();
    $myTimers['batch_processing']['start'] = 0;
    $refProperty-&gt;setValue(null, $myTimers);

  }


I now call revertTime() at the very end of all my "batch on crack" functions, juuuuust before they exit. It sets the start time of _process_batch()'s Timer::$timers['batch_processing']['start'] to 0.

And 0 is always more than one second ago.

The effect is that _batch_process() never loops; the Ajax call terminates after exactly one call to my batch function. 

10 ms later, the batch API initiates another Ajax call to my batch function, resulting potentially in a rat-tat-tat machine-gunning refresh cycle for my progress bar.

NOTE OF CAUTION: When I first ran this, with all checks and balances disabled, it overwhelmed Chrome with its rapid-fire Ajax calls every 10 ms. Today was a hot day (I live in the southern hemisphere) and my Mac slowed down to a crawl under the load of the Ajax calls... the built-in overheat protection kicked in!

When you use this hack, it is your responsibility to make your batch functions either do a LOT of slow stuff in one iteration, or waste sufficient time with usleep(). Otherwise the batch staccato may overwhelm your visitor's browser or your server.

Thanks again to all the contributors here and elsewhere who finally led me to a semi-clean (non-core hack) solution! ❤️