What was the greatest spatial extent of the Asteroid Belt prior to the Late Heavy Bombardment?
There have been some studies based on observations on the best preserved impact craters of that time in the inner solar system, namely those on Mercury and the Moon.  In the paper, Impact-induced compositional variations on Mercury (Edgard et al. 2014), state that based on the nature of Mercury's craters, they have developed a


  New dynamical simulations of the early evolution of the asteroid belt hint at the presence of additional
  asteroids in a region interior to the present-day belt, known as the “E-belt”.


Through their modelling and simulations, they have determined that the 'E-belt'


  The recent reevaluation of the early history of the inner Solar System suggests the primordial asteroid belt could have originally extended well into the Mars-crossing zone


The 'Hungarias' are thought to be a remnant of this belt - as shown in the diagram below



Image attribution: 
By Agmartin (Own work) [CC-BY-3.0], via Wikimedia Commons