Shift right by half a bit The challenge is to implement a program or function (subsequently referred to as "program") that takes a nonnegative integer \$n\$ as input and returns \$n\over\sqrt{2}\$ (the input divided by the square root of two) as output, rounded to a nonnegative integer.

You may take your input and output in any reasonable format; for example stdin/stdout, files, or arguments/return values would all be acceptable.

You are required to use, at minimum, the largest fixed-size integer type offered by your language, and if an unsigned variant of this is available, you must use it. If your language has no built-in integer type (e.g. JavaScript) you are allowed to use its default numerical type (e.g. floating point); for languages with no concept of a number (e.g. regex), input and output can be e.g. the length of a string.

It is not required to reject negative integers; a submission that returns correct answers for negative inputs is allowed, but not required. Undefined behavior with negative inputs is allowed.

You are allowed and encouraged to use arbitrary-precision integer types if you so desire, but the type must either be a built-in, part of a standard library, or implemented from scratch in your program. As such, there are two categories of competition in this challenge:


Precision limited by built-in types (fixed-size integer, floating point, etc.)
Arbitrary precision (recursion and memory allocation, if used, can be assumed to be unlimited, where appropriate to the language in question)


Despite what the title might imply, you may use any rounding algorithm you want (floor, ceiling, nearest half up, nearest half even, arbitrary, or even random), as long as the difference between the integer returned value and the theoretical exact (irrational) value is always less than \$1\$ for all inputs that fit in your chosen integer type (but exactly 0 for an input of 0). All inputs up to the maximum representable value must return a correct output.

In a way, the job of this program is to calculate the irrational number \$\sqrt{2}\$ to the requested precision, presenting it in the form of an integer. This is why solutions using arbitrary-precision types are encouraged, but not required.

This is a code-golf challenge. Standard loopholes are denied. The program with the least number of bytes wins. That said, this challenge is not only about which answer wins overall; it's also about seeing how concisely the challenge can be solved in each language, seeing how each language "prefers" to handle rounding, and how hard it is to solve in esoteric languages. And for those submissions that choose to use arbitrary precision, it's about seeing how concisely this can be done in the language.

Test cases

Within the Precision-limited category, only the cases that are in range of the language's capability are required to pass.

If your solution is too slow to demonstrably pass the larger inputs (or runs out of memory/stack), it becomes all the more important to explain it sufficiently well, so that it can be understood that it would pass.

Input → Floor or Ceiling
0 → 0 (this is the only input that can only result in one valid output)
1 → 0 or 1
2 → 1 or 2
3 → 2 or 3
4 → 2 or 3
5 → 3 or 4
10 → 7 or 8
32 → 22 or 23
500 → 353 or 354
1000 → 707 or 708
1000000 → 707106 or 707107
186444716 → 131836322 or 131836323
1000000000 → 707106781 or 707106782
2147483647 → 1518500249 or 1518500250
3037000499 → 2147483647 or 2147483648
4294967295 → 3037000499 or 3037000500
4503599627370496 → 3184525836262886 or 3184525836262887
9223372036854775807 → 6521908912666391105 or 6521908912666391106
18446744073709551615 → 13043817825332782211 or 13043817825332782212
10000000000000000000000000000000000000 → 7071067811865475244008443621048490392 or 7071067811865475244008443621048490393
956287480911872131784896176254337633353980911149964074434383 → 676197362516585909969655173274459790030028262421622111830069 or 676197362516585909969655173274459790030028262421622111830070
Regex (ECMAScript+(?*)), 1169 929 887 853 849 708 bytes
-141 bytes by using the second form of shortened division, where \$A^2 &gt; C\$
Regex was never designed to do mathematics. It has no concept of arithmetic. However, when input is taken in the form of bijective unary, as a sequence of identical characters in which the length represents a natural number, it is possible to do a wide range of operations, building up from the simple primitives available, which basically amount to addition, comparison, multiplication by a constant, and modulo. Everything must fit inside the input; it isn't possible to directly operate on numbers larger than that.
In ECMAScript regex, it's especially difficult (and therefore interesting) to do even some of the simplest of operations, because of the limitation that all backrefs captured in a loop are reset to empty at the beginning of each iteration – which makes it impossible to count anything directly. It's nevertheless possible to match prime numbers, powers of N, Nth powers, arbitrary multiplication and exponentiation, Fibonacci numbers, factorial numbers, abundant numbers, and more, much of which is demonstrated in my other answers.
One of the operations that turns out to be far more verbose than the rest is to &quot;calculate an irrational number&quot;. I initially discussed this with teukon back in 2014. The only known way to do this is to emulate operations on numbers larger than the input, and probably the simplest way to do this is by working in a number base chosen based on what can fit into the input.
It wasn't until late 2018 that I finally set about to implementing the theory I had sketched in 2014. Implementing it involved adapting the multiplication algorithm to work with factors of 0, which turned out to golf rather elegantly. (The underlying multiplication algorithm is explained in this post.) The basic algorithm is this:
For input \$N\$, we want to calculate \$M=\lfloor{N\over\sqrt2}\rfloor\$. So we want the largest \$M\$ such that \$2M^2\le N^2\$.
If we take the &quot;number base&quot; to be \$k=\lceil\sqrt N\rceil\$ or \$\lfloor\sqrt N\rfloor\!+\!1\$, all multiplication operations \$m\cdot n\$ on \$0\leq m,n&lt;k\$ are guaranteed to fit in the available space.
So if \$N=A k+B\$, where \$0\leq A,B\lt k\$, we can calculate \$N^2\$:
$$N^2=(A k+B)^2=A^2 k^2+2 A B k+B^2$$
We must then do division, modulo, and carry to bring \$A^2\$, \$2 A B\$, and \$B^2\$ back into the range of a base \$k\$ &quot;digit&quot;. A similar operation is then done to calculate \$2 M^2\$ iterated over the decreasing consecutive possible values of \$M\$, using digit-by-digit comparison to test for \$2M^2\le N^2\$, until the first \$M\$ is found that passes the test.
So while the basic concept is simple enough, it adds up to a lot of calculations, and the regex is huge! And this is probably the simplest calculation of an irrational number that can be done in ECMAScript regex. (It is still unknown whether it's possible to calculate a transcendental number to arbitrary precision in regex.)
This regex uses molecular lookahead, a.k.a. non-atomic lookahead, represented as (?*...). Without this feature, it would be much harder (or at least much more verbose) to implement.
A choice I made early on, to depart from pure code golf by going for mathematical aesthetics, turned out to be a very interesting. I chose to use \$k=\lceil\sqrt N\rceil\$ because it has the very neat property of making the calculations fit perfectly into \$N\$ if \$N\$ is a perfect square, whereas \$k=\lfloor\sqrt N\rfloor\!+\!1\$ is basically chaotic for all inputs. They both yield the same final outputs, but the former is just cleaner. A few golfs later, this choice ended up net increasing the total length of the regex by 8 bytes, so I figured it was worth it. (This change is in the git version history.) But another golf later that day, unbeknownst to me, was actually dependent on that decision! The skipping of a divisibility check in a division calculation makes \$N=25\$ return the incorrect output of \$M=11\$ instead of \$M=17\$ if \$k=\lfloor\sqrt N\rfloor\!+\!1\$, but works perfectly for all inputs if \$k=\lceil\sqrt N\rceil\$. So the actual net change in byte length was zero! It was a purely aesthetic decision for over two years.
At the time I did not understand why that division optimization worked, but this is now fully explained thanks to H.PWiz. The shortened form of division used at the beginning of the calculation of \$M^2\$ gives the correct quotient \$B\$ when \$A+2 &lt; 4B\$, where \$C\$ is the dividend and \$A\$ is the divisor. Previously I believed that it was only guaranteed to work when \$A \le B\$. [This is no longer used, due to the number base switch.]
And now, with the discovery of a second shortened form of division that gives the correct quotient when \$A^2 &gt; C\$, I found a major opportunity to use it in this regex! It only works when \$k=\lfloor\sqrt N\rfloor\!+\!1\$, so now I've switched back to that number base. It's saving 141 bytes! It's oddly convenient that this just happened to exist and works perfectly for this exact use.
(?=(x(x*)).*(?=\1*$)\2+$)(?=(x\1)+(x?(x*)))(?=\4(x(x*?))\1+$)(?=.*(?=(?=\4*$)\4\5+$)(x*?)(?=\3*$)(x?(x*?))(\1+$|$\9))(?=.*(?=(?=\4*$)(?=\6*$)(?=\4\7+$)\6\5+$|$\4)(x*?)(?=\3*$)(x?(x*?))(\1+$|$\13))(?=.*(?=\12\12\9$)(x*?)(?=\3*$)(x?(x*?))(\1+$|$\17))(?*.*?(?=((?=\3*(x?(x*)))\21(x(x*?))\1+$)))(?=.*(?=\23*$)(\23\24+$))(?=.*(?=(?=\21*$)\21\22+$)(x*?)(?=\3*$)(x?(x*?))(\1+$|$\27))(?=.*(?=(?=\21*$)(?=\23*$)(?=\21\24+$)\23\22+$|$\21)(x*?)(?=\3*$)(x?(x*?))(\1+$|$\31))(?=.*(?=\30\30\27$)(x*?)(?=\3*$)(x?(x*?))(\1+$|$\35))(?=.*(?=\26\26)(?=\3*(x*))(\1(x)|))(?=.*(?=\34\34\40)(?=\3*(x*))(\1(x)|))(?=(?=(.*)\13\13\17(?=\6*$)\6\7+$)\44(x+|(?=.*(?!\16)\41|(?!.*(?!\38)\8).*(?=\16$)\41$))(\25\31\31\35){2}\43$)\20|xx?\B|
Try it on repl.it
This regex is on GitHub with a full version history.
# Given an input number N in the domain ^x*$, this regex returns floor(N / sqrt(2))
(?=
    (x(x*))                    # \1 = will be the square root of the main number, rounded down; \2 = \1 - 1
    .*(?=\1*$)
    \2+$
)

# Step 1: Calculate N*N in base floor(sqrt(N))+1. Thanks to this choice of number base to work in, we'll be able to use the
# second shortened form of division in all places where the number base is the divisor, because it's guaranteed to give the
# correct quotient when the dividend is less than the squared divisor, and N itself is less than this. This form of division
# can be recognized by its lazy rather than greedy matching of the quotient, and only one divisibility test following that.

(?=(x\1)+(x?(x*)))             # \3 = \1+1 = floor(sqrt(N))+1, the number base to work in; \4 = N % \3; \5 = \4-1, or 0 if \4==0
(?=
    \4
    (x(x*?))                   # \6 = floor(N / \3); \7 = \6-1
    \1+$
)
(?=
    .*
    (?=
        (?=\4*$)               # tail = \4 * \4
        \4\5+$
    )
    (x*?)(?=\3*$)              # \8 =       (\4 * \4) % \3, the base-\3 digit in place 0 of the result for N*N
    (x?(x*?))                   # \9 = floor((\4 * \4) / \3); \10 = \9-1, or 0 if \9==0
    (
        \1+$
    |
        $\9                    # must make a special case for \9==0, because \1 is nonzero
    )
)
(?=
    .*
    (?=
        (?=\4*$)               # tail = \4 * \6; must do symmetric multiplication, because \4 is occasionally 1 larger than \6
        (?=\6*$)
        (?=\4\7+$)
           \6\5+$
    |
        $\4                    # must make a special case for \4==0, because \6 might not be 0
    )
    (x*?)(?=\3*$)              # \12 =       (\4 * \6) % \3
    (x?(x*?))                  # \13 = floor((\4 * \6) / \3); \14 = \13-1, or 0 if \13==0
    (
        \1+$
    |
        $\13                   # must make a special case for \13==0, because \1 is nonzero
    )
)
(?=
    .*(?=\12\12\9$)            # tail =       2 * \12 + \9
    (x*?)(?=\3*$)              # \16 =       (2 * \12 + \9) % \3, the base-\3 digit in place 1 of the result for N*N
    (x?(x*?))                  # \17 = floor((2 * \12 + \9) / \3); \18 = \17-1, or 0 if \17==0
    (
        \1+$
    |
        $\17                   # must make a special case for \17==0, because \1 is nonzero
    )
)                              # {\6*\6 + 2*\13 + \17} = the base-\3 digit in place 2 of the result for N*N, which is allowed to exceed \3 and will always do so;
                               # Note that it will be equal to N iff N is a perfect square, because of the choice of number base.

# Step 2: Find the largest M such that 2*M*M is not greater than N*N

# Step 2a: Calculate M*M in base \3
(?*
    .*?                        # Determine value of M with backtracking, starting with largest values first
    (?=
        (                      # \20 =       M
            (?=\3*(x?(x*)))\21 # \21 =       M % \3; \22 = \21-1, or 0 if \21==0
            (x(x*?))           # \23 = floor(M / \3); \24 = \23-1
            \1+$
        )
    )
)
(?=
    .*
    (?=\23*$)
    (\23\24+$)                 # \25 = \23 * \23
)
(?=
    .*
    (?=
        (?=\21*$)              # tail = \21 * \21
        \21\22+$
    )
    (x*?)(?=\3*$)              # \26 =       (\21 * \21) % \3, the base-\3 digit in place 0 of the result for M*M
    (x?(x*?))                  # \27 = floor((\21 * \21) / \3); \28 = \27-1, or 0 if \27==0
    (
        \1+$
    |
        $\27                   # must make a special case for \27==0, because \1 is nonzero
    )
)
(?=
    .*
    (?=
        (?=\21*$)              # tail = \21 * \23; must do symmetric multiplication, because \21 is occasionally 1 larger than \23
        (?=\23*$)
        (?=\21\24+$)
           \23\22+$
    |
        $\21                   # must make a special case for \21==0, because \23 might not be 0
    )
    (x*?)(?=\3*$)              # \30 =       (\21 * \23) % \3
    (x?(x*?))                  # \31 = floor((\21 * \23) / \3); \32 = \31-1, or 0 if \31==0
    (
        \1+$
    |
        $\31                   # must make a special case for \31==0, because \1 is nonzero
    )
)
(?=
    .*(?=\30\30\27$)           # tail =       2 * \30 + \27
    (x*?)(?=\3*$)              # \34 =       (2 * \30 + \27) % \3, the base-\3 digit in place 1 of the result for M*M
    (x?(x*?))                  # \35 = floor((2 * \30 + \27) / \3); \36 = \35-1, or 0 if \35==0
    (
        \1+$
    |
        $\35                   # must make a special case for \35==0, because \1 is nonzero
    )
)                              # {\25 + 2*\31 + \35} = the base-\3 digit in place 2 of the result for M*M, which is allowed to exceed \3 and will always do so

# Step 2b: Calculate 2*M*M in base \3
(?=
    .*
    (?=\26\26)                 # tail =       2*\26
    (?=\3*(x*))                # \38 =       (2*\26) % \3, the base-\3 digit in place 0 of the result for 2*M*M
    (\1(x)|)                   # \40 = floor((2*\26) / \3) == +1 carry if {2*\26} does not fit in a base \3 digit
)
(?=
    .*
    (?=\34\34\40)              # tail =       2*\34 + \40
    (?=\3*(x*))                # \41 =       (2*\34 + \40) % \3, the base-\3 digit in place 1 of the result for 2*M*M
    (\1(x)|)                   # \43 = floor((2*\34 + \40) / \3) == +1 carry if {2*\34 + \40} does not fit in a base \3 digit
)                              # 2*(\25 + 2*\31 + \35) + \43 = the base-\3 digit in place 2 of the result for 2*M*M, which is allowed to exceed \3 and will always do so

# Step 2c: Require that 2*M*M &lt;= N*N

(?=
    (?=
        (.*)                   # \44
        \13\13\17
        (?=\6*$)               # tail = \6 * \6
        \6\7+$
    )
    \44                        # tail = {\6*\6 + 2*\13 + \17}; we can do this unconditionally because our digits in place 2 are always greater than those in places 0..1
    (
        x+
    |
        (?=
            .*(?!\16)\41       # \41 &lt; \16
        |
            (?!.*(?!\38)\8)    # \38 &lt;= \8
            .*(?=\16$)\41$     # \41 == \16
        )
    )
    (\25\31\31\35){2}\43$      # 2*(\25 + 2*\31 + \35) + \43
)

\20

|xx?\B|                        # handle inputs in the domain ^x{0,4}$


Regex (ECMAScript 2018), 861 720 bytes
This is a direct port of the 849 708 byte molecular lookahead version, using variable length lookbehind.
(?=(x(x*)).*(?=\1*$)\2+$)(?=(x\1)+(x?(x*)))(?=\4(x(x*?))\1+$)(?=.*(?=(?=\4*$)\4\5+$)(x*?)(?=\3*$)(x?(x*?))(\1+$|$\9))(?=.*(?=(?=\4*$)(?=\6*$)(?=\4\7+$)\6\5+$|$\4)(x*?)(?=\3*$)(x?(x*?))(\1+$|$\13))(?=.*(?=\12\12\9$)(x*?)(?=\3*$)(x?(x*?))(\1+$|$\17))(?=.*?(?=((?=\3*(x?(x*)))\21(x(x*?))\1+$))(?&lt;=(?=(?=.*(?=\23*$)(\23\24+$))(?=.*(?=(?=\21*$)\21\22+$)(x*?)(?=\3*$)(x?(x*?))(\1+$|$\27))(?=.*(?=(?=\21*$)(?=\23*$)(?=\21\24+$)\23\22+$|$\21)(x*?)(?=\3*$)(x?(x*?))(\1+$|$\31))(?=.*(?=\30\30\27$)(x*?)(?=\3*$)(x?(x*?))(\1+$|$\35))(?=.*(?=\26\26)(?=\3*(x*))(\1(x)|))(?=.*(?=\34\34\40)(?=\3*(x*))(\1(x)|))(?=(?=(.*)\13\13\17(?=\6*$)\6\7+$)\44(x+|(?=.*(?!\16)\41|(?!.*(?!\38)\8).*(?=\16$)\41$))(\25\31\31\35){2}\43$))^.*))\20|xx?\B|
Try it online!
This regex is on GitHub.
# Given an input number N in the domain ^x*$, this regex returns floor(N / sqrt(2))
(?=
    (x(x*))                    # \1 = will be the square root of the main number, rounded down; \2 = \1 - 1
    .*(?=\1*$)
    \2+$
)

# Step 1: Calculate N*N in base floor(sqrt(N))+1. Thanks to this choice of number base to work in, we'll be able to use the
# second shortened form of division in all places where the number base is the divisor, because it's guaranteed to give the
# correct quotient when the dividend is less than the squared divisor, and N itself is less than this. This form of division
# can be recognized by its lazy rather than greedy matching of the quotient, and only one divisibility test following that.

(?=(x\1)+(x?(x*)))             # \3 = \1+1 = floor(sqrt(N))+1, the number base to work in; \4 = N % \3; \5 = \4-1, or 0 if \4==0
(?=
    \4
    (x(x*?))                   # \6 = floor(N / \3); \7 = \6-1
    \1+$
)
(?=
    .*
    (?=
        (?=\4*$)               # tail = \4 * \4
        \4\5+$
    )
    (x*?)(?=\3*$)              # \8 =       (\4 * \4) % \3, the base-\3 digit in place 0 of the result for N*N
    (x?(x*?))                  # \9 = floor((\4 * \4) / \3); \10 = \9-1, or 0 if \9==0
    (
        \1+$
    |
        $\9                    # must make a special case for \9==0, because \1 is nonzero
    )
)
(?=
    .*
    (?=
        (?=\4*$)               # tail = \4 * \6; must do symmetric multiplication, because \4 is occasionally 1 larger than \6
        (?=\6*$)
        (?=\4\7+$)
           \6\5+$
    |
        $\4                    # must make a special case for \4==0, because \6 might not be 0
    )
    (x*?)(?=\3*$)              # \12 =       (\4 * \6) % \3
    (x?(x*?))                  # \13 = floor((\4 * \6) / \3); \14 = \13-1, or 0 if \13==0
    (
        \1+$
    |
        $\13                   # must make a special case for \13==0, because \1 is nonzero
    )
)
(?=
    .*(?=\12\12\9$)            # tail =       2 * \12 + \9
    (x*?)(?=\3*$)              # \16 =       (2 * \12 + \9) % \3, the base-\3 digit in place 1 of the result for N*N
    (x?(x*?))                  # \17 = floor((2 * \12 + \9) / \3); \18 = \17-1, or 0 if \17==0
    (
        \1+$
    |
        $\17                   # must make a special case for \17==0, because \1 is nonzero
    )
)                              # {\6*\6 + 2*\13 + \17} = the base-\3 digit in place 2 of the result for N*N, which is allowed to exceed \3 and will always do so;
                               # Note that it will be equal to N iff N is a perfect square, because of the choice of number base.

# Step 2: Find the largest M such that 2*M*M is not greater than N*N

# Step 2a: Calculate M*M in base \3
(?=
    .*?                            # Determine value of M with backtracking, starting with largest values first
    (?=
        (                          # \20 =       M
            (?=\3*(x?(x*)))\21     # \21 =       M % \3; \22 = \21-1, or 0 if \21==0
            (x(x*?))               # \23 = floor(M / \3); \24 = \23-1
            \1+$
        )
    )
    (?&lt;=
        (?=
            (?=
                .*
                (?=\23*$)
                (\23\24+$)                 # \25 = \23 * \23
            )
            (?=
                .*
                (?=
                    (?=\21*$)              # tail = \21 * \21
                    \21\22+$
                )
                (x*?)(?=\3*$)              # \26 =       (\21 * \21) % \3, the base-\3 digit in place 0 of the result for M*M
                (x?(x*?))                  # \27 = floor((\21 * \21) / \3); \28 = \27-1, or 0 if \27==0
                (
                    \1+$
                |
                    $\27                   # must make a special case for \27==0, because \1 is nonzero
                )
            )
            (?=
                .*
                (?=
                    (?=\21*$)              # tail = \21 * \23; must do symmetric multiplication, because \21 is occasionally 1 larger than \23
                    (?=\23*$)
                    (?=\21\24+$)
                       \23\22+$
                |
                    $\21                   # must make a special case for \21==0, because \23 might not be 0
                )
                (x*?)(?=\3*$)              # \30 =       (\21 * \23) % \3
                (x?(x*?))                  # \31 = floor((\21 * \23) / \3); \32 = \31-1, or 0 if \31==0
                (
                    \1+$
                |
                    $\31                   # must make a special case for \31==0, because \1 is nonzero
                )
            )
            (?=
                .*(?=\30\30\27$)           # tail =       2 * \30 + \27
                (x*?)(?=\3*$)              # \34 =       (2 * \30 + \27) % \3, the base-\3 digit in place 1 of the result for M*M
                (x?(x*?))                  # \35 = floor((2 * \30 + \27) / \3); \36 = \35-1, or 0 if \35==0
                (
                    \1+$
                |
                    $\35                   # must make a special case for \35==0, because \1 is nonzero
                )
            )                              # {\25 + 2*\31 + \35} = the base-\3 digit in place 2 of the result for M*M, which is allowed to exceed \3 and will always do so

            # Step 2b: Calculate 2*M*M in base \3
            (?=
                .*
                (?=\26\26)                 # tail =       2*\26
                (?=\3*(x*))                # \38 =       (2*\26) % \3, the base-\3 digit in place 0 of the result for 2*M*M
                (\1(x)|)                   # \40 = floor((2*\26) / \3) == +1 carry if {2*\26} does not fit in a base \3 digit
            )
            (?=
                .*
                (?=\34\34\40)              # tail =       2*\34 + \40
                (?=\3*(x*))                # \41 =       (2*\34 + \40) % \3, the base-\3 digit in place 1 of the result for 2*M*M
                (\1(x)|)                   # \43 = floor((2*\34 + \40) / \3) == +1 carry if {2*\34 + \40} does not fit in a base \3 digit
            )                              # 2*(\25 + 2*\31 + \35) + \43 = the base-\3 digit in place 2 of the result for 2*M*M, which is allowed to exceed \3 and will always do so

            # Step 2c: Require that 2*M*M &lt;= N*N

            (?=
                (?=
                    (.*)                   # \44
                    \13\13\17
                    (?=\6*$)               # tail = \6 * \6
                    \6\7+$
                )
                \44                        # tail = {\6*\6 + 2*\13 + \17}; we can do this unconditionally because our digits in place 2 are always greater than those in places 0..1
                (
                    x+
                |
                    (?=
                        .*(?!\16)\41       # \41 &lt; \16
                    |
                        (?!.*(?!\38)\8)    # \38 &lt;= \8
                        .*(?=\16$)\41$     # \41 == \16
                    )
                )
                (\25\31\31\35){2}\43$      # 2*(\25 + 2*\31 + \35) + \43
            )
        )
        ^.*
    )
)
\20
|xx?\B|                        # handle inputs in the domain ^x{0,4}$


Regex (ECMAScript)
I have not yet ported this algorithm to basic ECMAScript. One way to do it would be to use \$k=\lceil\sqrt[\uproot{1}3]N\rceil\$ as the number base, and calculate:
$$N^2=(A k^2+B k+C)^2=A^2 k^4 + 2 A B k^3 + (2 A C + B^2)k^2 + 2 B C k + C^2$$
Another way would be to stick with \$k=\lceil\sqrt N\rceil\$, capture \$M\$ encoded into two or more backrefs, and emulate the existing calculations within the smaller available space. I'm not sure which way would be more concise. Either way, I expect the regex would roughly double in length.