Can an existing transformer model be modified to estimate the next most probable number in a sequence of numbers?
To answer this, you need some constraints on the problem. Here are some sequences of numbers. No machine learning technique could be expected to learn all of them:

the odd numbers
the primes
numbers expressed in digits, but listed in alphabetical order of their name in German
numbers listed in the lexical order of the reverse of their representation in base 3
the phone numbers in the Manhattan phone director, listed in alphabetical order of subscriber