Render array only working with numeric keys
The return value of FormatterInterface::viewElements is defined as:


  Return value
  
  array A renderable array for $items, as an array of child elements keyed by consecutive numeric indexes starting from 0.


The theming layer can't handle other arrays for field values, see  How to print all values of a multi-value field, except the first?