How Babbage's Analytic engine made calculations of unlimited extent?
tl;dr-  The bit about "unlimited extent" is referring to being able to perform general-purpose computation rather than infinite computation.  They probably thought that it was trippy for a finite device to be able to pursue any sort of computation.



It seems that Brian Randell was referring to how the Analytical Engine was meant to be a general-computing device that could, in principle, perform any sort of computation.

The fuller quote discusses the notions of Turing completeness and "unlimited extent" as two different perspectives on this topic:


  Turing thus was the first to arrive at an understanding of the universal nature of a (conceptual) digital computer which matches and indeed surpasses the philosophic understanding that I believe Babbage had attained, a century earlier, of the universality of his planned (mechanical) Analytical Engine. Babbage’s phrasing was “that the whole of the conditions which enable a finite machine to make calculations of unlimited extent are fulfilled in the Analytical Engine” [2, p.28], where the term “extent” encompassed both the amount and accuracy of the data to be processed, and the length and logical complexity of the algorithm to be performed. Central to the Universal Turing Machine is the idea of having data, and input data in particular, represent a program (called a “table”
  in Turing’s paper). A hitherto little known manuscript by Babbage [1] which has recently been published for the first time makes it clear that Babbage had reached an almost similar level of understanding. In the manuscript he points out that a fully detailed sequence of “formula cards” might be prepared by the Analytical Engine from a more abstract sequence. However, this is not to say that Turing’s work was in any way derived from Babbage’s – indeed there is no evidence that Turing even knew of Babbage at this time, but this topic will be returned to later.
  
  -"The COLOSSUS", Brian Randell, page 4


This seems to be referring to the property of the Analytical Engine to perform general-purpose computation, which the above quote refers to as "universal nature" and "universality".