Walking into the complex plane to "order" better time integrators

Abstract: Most numerical methods for time integration use real time steps. Complex time steps provide an additional degree of freedom, as we can select the magnitude of the step in both the real and imaginary directions. By time stepping along specific paths in the complex plane, integrators can gain higher orders of accuracy or achieve expanded stability regions. We show how to derive these paths for explicit and implicit methods, discuss computational costs and storage benefits, and demonstrate clear advantages for complex-valued systems like the Schrodinger equation. We also explore how complex time stepping also allows us to break the Runge-Kutta order barrier, enabling 5th order accuracy using only five function evaluations for real-valued differential equations.

\section{Introduction}

The real line is a tiny sliver in the whole of the complex plane. Wandering into the complex plane has often been employed to solve real problems. Two direct examples are the solution of complicated real-valued integrals via contour integrals \cite{brown2009complex} and the usage of the intuition gained from poles in the complex plane in control-theory applications \cite{nise2020control}. The complex plane provides wonderful insights to real problems, allowing mathematicians to impact practical applications.  One of the motivations for this paper was a simple numerical application  highlighted in a 2018 SIAM News article \cite{high18-diff} where Nick Higham describes how taking a numerical complex derivative neatly gets rid of round-off error. However, most methods for numerically solving a differential equation from time $a$ to time $b$, construct a path utilizing only the real line. We show that we can use the complex plane to our advantage when it comes to numerical time integration by taking time steps in the complex plane.

This paper is not the first to talk about complex time steps. Early work on time stepping in the complex plane focused on systematically avoiding singularities during numerical integration of singular differential equations \cite{corliss1980integrating} and as techniques to achieve superconvergence for Runge-Kutta methods \cite{orendt2009geometry}. A use of complex time steps most similar to our work was in \cite{chambers2003symplectic}, where Chambers explores operator splitting in symplectic integrators using complex substeps. Taking real substeps often results in negative substeps and substeps larger than the original step causing the symplectic integrator go back and forth along the direction of integration. By using complex substeps, Chambers showed how to avoid negative and unstable substeps, significantly reducing the truncation error. Chambers's work inspired others to consider complex time steps as part of operator splitting methods for various parabolic and nonlinear evolution problems \cite{castella2009splitting,auzinger2017practical,casas2021compositions,blanes2010splitting, hansen2009high}. Concurrent with our work, a very recent paper \cite{casas2021high} in this area uses complex coefficients for high order composition methods. Another recent paper develops time integration schemes using nodes in the complex plane based on Cauchy's integral formula and using the complex nodes to  facilitate parallel BDF formulation \cite{buvoli2019constructing}. Rather than focusing on a specific application or numerical scheme as has been done previously, we ask fundamental questions about the utility of numerical time-stepping in the complex plane, including: `What paths lead to the least error?' and `Are there general strategies and trade offs for complex time-stepping?'. Here we show that stepping into the complex plane provides an extra dimension which can be exploited to improve accuracy, design expanded stability regions and even break through the order barrier \cite{Butcher:2007, butcher2009fifth}. We compare against existing Runge-Kutta methods and  demonstrate improvement for particular proof of concept cases. Finally we analyze increased operation counts due to complex operations, potential storage benefits, extensions to stiff solvers, and strong stability preserving properties. The code to reproduce the numerical experiments in this paper is available on GitHub \footnote{\url{https://github.com/Dirivian/complex_time_integrators}} .

\section{Taking complex time steps with the Euler method}

  The idea of a complex time step can be unusual and unintuitive. To make it accessible, we first illustrate simple paths with complex time steps and demonstrate that applying the forward Euler method along the complex paths improves accuracy compared to forward Euler with real steps. The paths we choose in the complex plane must intersect the real line at the time points where we desire a solution. Our primary question is `what is the optimal n-step path in the complex plane over which to integrate between two real time points?'

To illustrate the simplest possible case, suppose we want to solve the equation $\dot{y} = \lambda y$ with the initial condition $y_0$ at time $t=0$. To find the solution $y(\Delta t)$, we can either take two real time steps of length $\frac{\Delta t}{2}$ or take a complex time step of length $\frac{\Delta t}{2} + i\frac{\Delta t}{2}$ followed by a complex time step of length $\frac{\Delta t}{2} - i\frac{\Delta t}{2}$ returning back to the real line. These two paths are shown in Fig. \ref{fig_1} (top). We numerically integrate the differential equations $\dot{y} = y$ along these paths as shown in Fig. \ref{fig_1} (bottom). By eye, the complex time path clearly does much better than the real time path in this linear case. The improvement with a complex path can hold for nonlinear ODEs, second order ODEs, non-autonomous ODEs (Fig. \ref{airyetall}) and nonlinear PDEs, such as the viscous Burgers equation (Fig. \ref{burgers}). In section \ref{higherordersection}, we describe the generalizations needed to solve nonlinear equations, but first we provide theoretical insight into the simple linear case.

\begin{figure}
\label{fig_1}
\begin{centering}
\includegraphics[width=5.3in]{sec2_1_v2.pdf}
\caption{Taking complex time steps that return to the real line results in increased accuracy.}
\end{centering}
\end{figure}

\begin{figure}
\label{airyetall}
\begin{centering}
\includegraphics[width=5.2in]{sec2_2_v2.pdf}
\caption{Comparison of real-valued classic Euler method and 2-step complex-valued Euler method. The number of real timepoints for the classic scheme are roughly double, $2n-1$, the number for the 2-step complex scheme so that the net number of timesteps and function evaluations are the same for both schemes.}
\end{centering}
\end{figure}

\begin{figure}
\label{burgers}
\begin{centering}
\includegraphics[width=5.1in]{burgers_error_v3_1.pdf}
\caption{The complex time steps reduce the error for the nonlinear viscous Burgers equation.}
\end{centering}
\end{figure}

Why does time stepping on the complex path improve accuracy of the numerical solution so significantly? To answer that, we analyze the linear problem $\dot{y} = y$ with initial condition $y(t_0) = y_0$. Consider time stepping from the initial condition $y_0$ at time $t_0$ to time $t+\Delta t$ using 2 real time steps. The discrete values of $y(t)$ are given by the following at $t = t_0, \; t_0+\frac{\Delta t}{2},$ and $t_0 + \Delta t$:

\begin{align}
    y(t_0) &= y_0 \nonumber\\
    y(t_0 + \frac{\Delta t}{2}) &= y_0 \bigg(1+\frac{\Delta t}{2} \bigg)\nonumber\\
     y(t_0 + \Delta t) &= y_0\bigg(1+\frac{\Delta t}{2}\bigg)\bigg(1+\frac{\Delta t}{2}\bigg) = y_0 \bigg(1 + \Delta t + \frac{\Delta t^2}{4}\bigg).
\end{align}

Now consider timestepping from $y_0$ at time $t_0$ to time $t+\Delta t$ using the two complex time steps.

\begin{align}
    y(t_0) &= y_0 \nonumber\\
    y\Big(t_0 + \frac{\Delta t}{2} +i\frac{\Delta t}{2}\Big) &= y_0 \bigg(1+\frac{\Delta t}{2}+i\frac{\Delta t}{2} \bigg)\nonumber\\
     y(t_0 + \Delta t) &= y_0\bigg(1+\frac{\Delta t}{2}+i\frac{\Delta t}{2}\bigg)\bigg(1+\frac{\Delta t}{2}-i\frac{\Delta t}{2}\bigg ) = y_0 \bigg(1 + \Delta t + \frac{\Delta t^2}{2}\bigg)
\end{align}

The exact solution to the problem is given by the Taylor expansion
\begin{align}
    y(t_0 +\Delta t) = y_0 e^{\Delta t} = y_0\Big(1+\Delta t + \frac{\Delta t^2}{2} + \hdots \Big).
\end{align}

The secret behind the higher accuracy of the complex 2-step method is that the polynomial obtained through the numerical approximation matches the Taylor series of the exact solution to the second order term, making forward Euler method gain second order accuracy. This is one of two unique 2-step paths in the complex plane which has this property. The other unique path uses the other permutation of the complex time steps $(\frac{\Delta t}{2} -i\frac{\Delta t}{2}, \frac{\Delta t}{2} +i\frac{\Delta t}{2})$ instead of $(\frac{\Delta t}{2} +i\frac{\Delta t}{2}, \frac{\Delta t}{2} -i\frac{\Delta t}{2})$. Both permutations of these steps will lead to the same polynomial with coefficients that match the Taylor expansion to 2nd order.

By stepping into the complex plane, we gain an extra degree of freedom in the coefficients that allows us to design time-steps with improved accuracy. This extra dimension can be used to construct higher order paths for linear and nonlinear differential equations (Section \ref{higherordersection}), design improved stability (Section \ref{stabilitysection}) and even break through the order barrier for Runge-Kutta methods (Section \ref{subsec:orderbarrier}). The paths can be combined with other methods, such as the implicit midpoint method (Runge-Kutta-2) as demonstrated in Section \ref{implicitsection}. In the next section, we demonstrate how to extend the linear example to construct paths with higher order accuracy.

\section{Constructing a higher order time integrator} \label{higherordersection}

Is it possible then to construct 3-step path on which forward Euler would be 3rd order accurate? Would it be nth order accurate on an n-step complex path?
To build intuition for the simplest higher order construction, we continue analyzing the linear problem in Subsection \ref{subsec:linearFEpaths}, and then prove that achieving higher order requires complex time-stepping. In Subsection \ref{subsec:nonlinearde}, we extend the construction of complex time-steppers to the general nonlinear problem.

\subsection{Higher order integrators for linear differential equations} \label{subsec:linearFEpaths}

In this subsection, we show how to construct a third-order time integrator for the ode $\dot{y} = \lambda y$ by taking 3 forward Euler time steps of variable sizes in the complex plane that return to the real line at every desired time point.

If we take the three complex steps $w_1 \Delta t, w_2 \Delta t$ and $w_3 \Delta t$ to timestep from $y_0$ at $t_0$ to $t_0 +\Delta t$, the approximate numerical solution $\bar{y}$ at time $t +\Delta t$ is given by

\begin{align}
    \bar{y}(t_0 +\Delta t) = y_0(1+w_{1} \lambda \Delta t)(1+w_{2} \lambda \Delta t)(1+w_3\lambda \Delta t)
\end{align}
where the $w_{i}$'s are complex variables. Collecting $\lambda \Delta t$ terms results in
\begin{align}
\label{eq:linearcoeff}
\bar{y}(t_0 +\Delta t)
&= y_0(1 + (w_1 +w_2 +w_3)\lambda \Delta t + (w_1w_2 + w_2w_3 + w_1w_3) \lambda^2 \Delta t^2  + w_1w_2w_3 \lambda^3 \Delta t^3).
\end{align}
The exact solution is given by
\begin{align}
\label{eq:expcoeff}
y(t_0 +\Delta t)= y_0e^{\lambda \Delta t} = y_0(1 + \lambda \Delta t + \frac{1}{2} \lambda^2 \Delta t^2 +\frac{1}{6} \lambda^3 \Delta t^3 + \hdots )
\end{align}

For the numerical approximation to be third-order accurate, the coefficients of $y_0(\lambda \Delta t)^n$ for $n=1,2,3$ in equations \eqref{eq:linearcoeff} and \eqref{eq:expcoeff} need to match. This results in the following order conditions:

\begin{align}
\label{nonlinearsystem1}
    w_1 + w_2 +w_3 &= 1 \nonumber\\
    w_1w_2 + w_2w_3 + w_1w_3 &= \frac{1}{2} \nonumber\\
    w_1w_2w_3 &= \frac{1}{6}.
\end{align}

One of the solutions to the above nonlinear system is
\begin{align}
    (w_1, w_2, w_3) = (0.186731 + 0.480774 i, 0.626538, 0.186731 - 0.480774i).
\end{align}
However, as the nonlinear system \ref{nonlinearsystem1} is symmetric about $w_1, w_2$ and $w_3$, all 6 permutations of $(0.186731 + 0.480774 i, 0.626538, 0.186731 - 0.480774i)$ result in unique 3-step 3rd order complex time stepping paths. In Fig \ref{fig:pathslinear}, all six 3-step 3rd order paths are shown along with the two 2-step 2nd order paths and the single step forward Euler path.

For a general n-step nth order path, the order conditions would be

\begin{align}
    \sum_i w_i &= 1 \nonumber\\
   \sum_{i \neq j} w_iw_j &= \frac{1}{2} \nonumber\\
    \sum_{i \neq j \neq k} w_iw_j w_k &= \frac{1}{6} \nonumber\\
   & \vdots \nonumber\\
  \prod_{i=1}^{n}w_i &=\frac{1}{n!}.
\end{align}

For higher orders, the solutions take on a complicated form and solving the nonlinear system becomes a tough symbolic task. However, we only need to find one solution for high-order approximations and such these solution can be found numerically and be stored in a library for general use.

 \begin{figure}[H]
\begin{centering}
\includegraphics[width=5.2in]{3_1_v3_ink.pdf}
\caption{All possible first, second, and third-order complex Euler methods for linear differential equations. Each order requires an additional step and adds an additional constraint on the coefficients.}
\end{centering}
\label{fig:pathslinear}
\end{figure}

These results beg the theoretical question: is there any way to take steps of varying size on the real line and achieve a higher-order time integrator? or is it only possible to do so by going into the complex plane?
\begin{theorem}

It is impossible to construct a time integrator with order greater than or equal to 2 using real-valued Euler time steps of variable sizes.
\end{theorem}

\begin{proof}
We prove the above theorem by showing that it is impossible to construct a second-order time integrator using forward Euler steps of variable size on the real line.

Suppose we use $n$ time steps of variable size given by $w_1, w_2, \hdots, w_n$. The approximate solution to $\dot{y} = \lambda y$ given by our time integrator $\bar{y}$ would be

\[\bar{y}(\Delta t) = (1+w_1 \lambda \Delta t)(1+w_2 \Delta \lambda t)\hdots(1+w_n \Delta \lambda t)  \]
where we assume the first order condition is satisfied,
\[w_1 +w_2 + \hdots +w_n = 1.\]
Gathering together different order $\lambda \Delta t$ terms,
\begin{align*}
    \bar{y}(\Delta t) &= 1 + (w_1 +w_2 + \hdots +w_n)\lambda \Delta t +  (w_1w_2+w_1w_3 +\hdots w_{n-1}w_n) \lambda^2 \Delta t^2 +\hdots\\
     &= 1 + \lambda \Delta t +  (w_1w_2+w_1w_3 +\hdots w_{n-1}w_n) \lambda^2 \Delta t^2 +\hdots \\
\end{align*}
where we have substituted in our first order condition. For the time stepper to be second order, we need the coefficient on $\lambda^2 \Delta t^2$to be

\[w_1w_2+w_1w_3 +\hdots+ w_2w_3 + \hdots+ w_1w_n + w_2w_n + \hdots+ w_{n-1}w_n = \frac{1}{2}.\]
Factoring out a $w_n$ results in
\[w_1w_2+w_1w_3 +\hdots+ w_2w_3 + \hdots+  (w_1 +w_2 + \hdots +w_{n-1})w_n = \frac{1}{2}.\]
Solving the first order condition for $w_n$ and incorporating produces
\[w_1w_2+w_1w_3 +\hdots+ w_2w_3 + \hdots+  (w_1 +w_2 + \hdots +w_{n-1})(1- w_1 -w_2  \hdots -w_{n-1}) = \frac{1}{2},\]
which rearranges to
\[w_1 +w_2 + \hdots +w_{n-1} - w_1^2 - w_2^2\hdots -w_{n-1}^2- w_1w_2-w_1w_3 \hdots- w_{n-1}w_{n-2}= \frac{1}{2}.\]
Since all the $w$s are real, we can express them as
\[w_j = k_j w_1 \quad j = 1, \hdots, n-1\]
where all the $k$s are real. Then, we get
\[(k_1 +k_2 + \hdots +k_{n-1})w_1 - w_1^2(k_1^2 + k_2^2\hdots +k_{n-1}^2+ k_1k_2+k_1k_3 \hdots+ k_{n-1}k_{n-2})= \frac{1}{2}.\]

Rearranging in terms of powers of $w_1$ one has
\[ w_1^2(k_1^2 + k_2^2\hdots +k_{n-1}^2+ k_1k_2+k_1k_3 \hdots+ k_{n-1}k_{n-2})-(k_1 +k_2 + \hdots +k_{n-1})w_1 + \frac{1}{2} = 0.\]

This is quadratic equation to solve for $w_1$ in the form of $ax^2+bx+c = 0$. For $w_1$ to only have real solutions, we need a positive discriminant $\Delta = b^2-4ac \geq 0 $. In  our case, the discriminant, $\Delta$, is

\begin{align*}
    \Delta &= (k_1 +k_2 + \hdots +k_{n-1})^2 - 4 (k_1^2 + k_2^2\hdots +k_{n-1}^2+ k_1k_2+k_1k_3 \hdots+ k_{n-1}k_{n-2}) \frac{1}{2} \nonumber\\ &= - (k_1^2 + k_2^2 + \hdots +k_{n-1}^2)
\end{align*}
Since the discriminant is always negative, $w_1$ cannot be a purely real time step and must to be complex to satisfy our second order condition. Therefore, not only \textit{can} we achieve higher order integrators through complex time stepping, but we \textit{must} step into the complex plane to achieve higher order.

\end{proof}

\subsection{Higher order integrators for general nonlinear non-autonomous differential equations}\label{subsec:nonlinearde}

In the previous subsection, we showed how to construct a 3-step 3rd order complex path on which to time integrate using forward Euler steps. We can experimentally demonstrate that the order of accuracy is obeyed by every linear ordinary differential equation and partial differential equation that we have tested it on. However, the derivation assumed a linear problem and does not guarantee the same order of accuracy for nonlinear problems. Indeed, when we tested the complex time-stepping schemes on nonlinear and non-autonomous equation, the 3 step paths achieved only second order accuracy. To achieve higher order accuracy for nonlinear, non-autonomous equations, we generalize the procedure we described so far to obtain 3rd order accuracy.

A general nonlinear ode is given by

\begin{align}
    \dot{y} = f(t,y), \quad y(t_0) = y_0.
\end{align}

Let us assume the function $f$ is sufficiently analytic near the region of integration i.e it does not have any singularities close to the path of integration. At time $t = t_0 + \Delta t$, we have

\begin{align}
\label{taylornonlinexp}
    y(t_0 + \Delta t) = y_0 + \Delta t f(t_0, y_0) + \frac{1}{2} \Delta t^2 \dot{f}(t_0,y_0) + \frac{1}{3!} \Delta t^3 \ddot{f}(t_0,y_0)  + \hdots
\end{align}

where
\begin{align}
    \dot{f}(t,y) = \frac{d}{dt}f(t,y)\at[\Bigg]{t=t_0, y = y_0} = f{\left(t_0,y_0 \right)} \frac{\partial}{\partial y} f{\left(t,y \right)}\at[\Bigg]{t=t_0, y = y_0} + \frac{\partial}{\partial t} f{\left(t,y \right)\at[\Bigg]{t=t_0, y = y_0}}.
\end{align}

Expanding equation \ref{taylornonlinexp}, we get

\begin{align}
\label{nonlintay}
    y(t_0+\Delta t) = &y_0 + \Delta t f(t_0, y_0) + \frac{1}{2} \Delta t^2 \bigg( f{\left(t,y \right)} \frac{\partial}{\partial y} f{\left(t,y \right)} + \frac{\partial}{\partial t} f{\left(t,y \right)}
\bigg)\at[\Bigg]{t=t_0, y = y_0}  \nonumber \\
    & + \frac{1}{3!} \Delta t^3 \bigg(f^2{\left(t,y \right)} \frac{\partial^{2}}{\partial y^{2}} f{\left(t,y \right)} + \left(\frac{\partial}{\partial y} f{\left(t,y \right)}\right)^{2} f{\left(t,y \right)}\nonumber \\ &+ 2f{\left(t,y \right)} \frac{\partial^{2}}{\partial y\partial t} f{\left(t,y \right)} + \frac{\partial}{\partial t} f{\left(t,y \right)} \frac{\partial}{\partial y} f{\left(t,y \right)} + \frac{\partial^{2}}{\partial t^{2}} f{\left(t,y \right)} \bigg)\at[\Bigg]{t=t_0, y = y_0}\\
    &+ \hdots  \nonumber
\end{align}

Now, consider a complex time stepper that takes 3 complex time steps $w_1 \Delta t, w_2 \Delta t, w_3 \Delta t $ starting from $y_0$ to go to $y_1, y_2, y_3$. These steps are are given by:

\begin{align}
    y_0 &= y(t_0)\\ \label{nonlintay1}
    y_1 &= y_0 + w_1 \Delta t f( t_0, y_0)\\
        y_2 &= y_1 + w_2 \Delta t f( t_0 + w_1 \Delta t, y_1)\\
    y_3 &= y_2 + w_3 \Delta t f( t_0 + w_1 \Delta t+ w_2 \Delta t, y_2) \label{nonlintay4}.
\end{align}

One can expand $f(t_n, y_n)$ around $f(t_0,y_0)$ using the multivariate Taylor series. Doing so,

\begin{align}\label{3comptay}
    y_3 = &y_0 + \Delta t (w_{1}+w_{2}+w_{3}) f{\left(t_0,y_0 \right)} \nonumber\\&+  \Delta t^{2} (w_1 w_2 + w_1 w_3 +w_2 w_3)\bigg(f{\left(t,y \right)} \frac{\partial}{\partial y} f{\left(t,y \right)} +  \frac{\partial}{\partial t} f{\left(t,y \right)}  \bigg)\at[\Bigg]{t=t_0, y = y_0} \nonumber\\&
    +  \Delta t^{3} \bigg( \frac{1}{2} (w_{1}^{2} w_{2}+ w_{1}^{2} w_{2}+ 2w_1 w_2 w_3 + w_{2}^{2} w_{3}) f^{2}{\left(t,y \right)} \frac{\partial^{2}}{\partial y^{2}} f{\left(t,y \right)}  \nonumber\\& + w_{1} w_{2} w_{3} f{\left(t,y \right)} \left(\frac{\partial}{\partial y} f{\left(t,y \right)}\right)^{2} + w_{1} w_{2} w_{3} \frac{\partial}{\partial t} f{\left(t,y \right)} \frac{\partial}{\partial y} f{\left(t,y \right)} \nonumber\\& +  (w_{1}^{2} w_{2}+ w_{1}^{2} w_{2}+ 2w_1 w_2 w_3 + w_{2}^{2} w_{3}) f{\left(t,y \right)} \frac{\partial^{2}}{\partial y\partial t} f{\left(t,y \right)} \nonumber\\&+ \frac{1}{2} (w_{1}^{2} w_{2}+ w_{1}^{2} w_{2}+ 2w_1 w_2 w_3 + w_{2}^{2} w_{3}) \frac{\partial^{2}}{\partial t^{2}} f{\left(t,y \right)}  \bigg)\at[\Bigg]{t=t_0, y = y_0} + O\left(\Delta t^{4}\right).
\end{align}

For this three step complex integrator to have 3rd order accuracy, the coefficients of the partial derivatives in Eq \ref{3comptay} should match that of Eq \ref{nonlintay}. Equating coefficients, we get this system of nonlinear equations.

\begin{align}
    \label{syseq}
    w_1 +w_2 +w_3 = 1 \nonumber\\
    w_1w_2 + w_1w_3 + w_2w_3 = \frac{1}{2} \nonumber\\
       w_1 w_2 w_3 = \frac{1}{6} \nonumber\\
        w_{1}^{2} w_{2} +  w_{1}^{2} w_{3}  + 2 w_{1} w_{2} w_{3}  +  w_{2}^{2} w_{3}  = \frac{2}{3}
\end{align}

Unfortunately, this system is overconstrained and does not have a solution, implying that we may need more than 3 steps for 3rd order accuracy. However, for systems with only real solutions we can take further advantage of the degree of freedom in the imaginary part of the step and find many 3-step paths with 3rd order accuracy. If we allow the truncation error at the order of $\Delta t^3$ to be purely imaginary, we can effectively remove the error at the end of the 3 step sequence by taking only the real part of the numerical solution at the end of the complex path. Let $y$ be the true solution and $\bar{y}$ be the approximate numerical solution. Then, with our setup, we would have

\begin{align}
    y(t+\Delta t) = \bar{y}(t+\Delta t)+ ik\Delta t^3  + O(\Delta t^4)
\end{align}
where $k$ is real.
Taking the real part on both sides, we get
\begin{align}
    \textbf{Re}(y(t+\Delta t)) = \textbf{Re}(\bar{y}(t+\Delta t))  + O(\Delta t^4)
\end{align}

If the true solution to our differential equation and its initial condition are known to be real-valued at real time points, then we have $y(t+\Delta t) =  \textbf{Re}(y(t+\Delta t))$ and taking the real part of the solution at the end of complex path results in 3rd order accurate numerical solution. This formulation, allowing the error to be purely imaginary at $O(\Delta t^3)$ has fewer constraints on the coefficients, which must now satisfy the following relaxed conditions:

\begin{align}
    \label{syseq2}
    w_1 +w_2 +w_3 = 1 \nonumber\\
    w_1w_2 + w_1w_3 + w_2w_3 = \frac{1}{2} \nonumber\\
       \textbf{Re}(w_1 w_2 w_3) = \frac{1}{6} \nonumber\\
        \textbf{Re}(w_{1}^{2} w_{2} +  w_{1}^{2} w_{3}  + 2 w_{1} w_{2} w_{3}  +  w_{2}^{2} w_{3})  = \frac{2}{3}.
\end{align}

There are many paths satisfying these equations. Of the 3rd order paths in Fig \ref{fig:pathslinear} that were found to satisfy the linear order conditions, the top and bottom (cyan and brown) paths also satisfy the 3rd order nonlinear  order conditions. We ran convergence tests for various nonlinear problems listed in Section \ref{subsec:numericalresults}, and demonstrated 3rd order accuracy with those paths (Fig \ref{convergence_all}).

In this subsection, we have derived paths in the complex plane that satisfy the order conditions for scalar non-autonomous differential equations. A natural follow-up question is whether these paths return the same order for vector valued non-autonomous differential equations, expanding our result to systems of differential equations and partial differential equations. Butcher showed in \cite{butcher2009fifth} that the order conditions for scalar non-autonomous differential equations are the same as those for vector valued differential equations up to order 4. As the paths derived here are of order 3 or less, they hold for systems of differential equations and partial differential equations as seen in the next subsection. To construct a fifth order or higher complex path, we would need extract the necessary order conditions using Butcher's rooted tree approach \cite{butcher2009fifth}.

\subsection{Numerical Results}\label{subsec:numericalresults}

We have tested our complex time-stepping schemes on a variety of differential equations to show that they satisfy the order conditions. The equations are listed in Table \ref{DElist} along with a unique label corresponding to their legend on the convergence plot in Fig.\ref{convergence_all}.
  \begin{figure}[H]
\begin{centering}
\includegraphics[width=6in]{convergence_all_v2.pdf}
\caption{Convergence of various ODEs and PDEs (Sec. \ref{subsec:nonlinearde}) for 1-step (forward Euler), 2-step and 3-step complex time steps. The path/method used is displayed above the corresponding convergence plot. The orders of accuracy are as expected.}
 \label{convergence_all}
\end{centering}

\end{figure}

\begin{table}
\label{DElist}
\caption{List of differential equations}
\begin{tabularx}{\textwidth}{|>{\setlength\hsize{\hsize}\setlength\linewidth{\hsize}}X|}
\hline
1. Linear ODEs
\begin{itemize}
\item The Dahlquist test problem (`linear')
\begin{align*}
    \dot{y} = \lambda y,\quad y(0) = 1
\end{align*}
\item  Simple harmonic oscillator (`shm')
    \begin{align*}
    \ddot{y} = -y, \quad y(0) = 1, \quad y'(0) = 0
\end{align*}
\end{itemize}
\vphantom{1. Real unequal eigenvalues of same sign}
\\
\hline
2. Nonlinear ODEs
\begin{itemize}
    \item Nonlinear autonomous ode 1(`square')
\begin{align*}
    \dot{y} = - y^2, \quad y(0) = 1
\end{align*}

    \item Nonlinear autonomous ode 2 (`exp')
\begin{align*}
    \dot{y} = -e^y, \quad y(0) = 1
\end{align*}

    \item Nonlinear sine(`nlsin')
\begin{align*}
    \dot{y} = 4y\sin{(t)}^3\cos{(t)}, \quad y(0) = 1
\end{align*}
\item  The Van Der Pol Oscillator (vdp")
\begin{align*}
    \dot{y_1} = y_2, \dot{y_2} = \mu(1-y_1^2)y_2-y_1, \quad (y_1, y_2)(0) = (2,0)
\end{align*}
\end{itemize}
\vphantom{2. Real unequal eigenvalues of opposite sign}
\\
\hline
3. Linear PDE : The advection equation (`wave')

 This is a linear PDE given by
 \begin{align}
     u_{t} = cu_x, \quad x \in (0,2\pi), \quad t \in (0, 1)
 \end{align}

 This PDE is solved with periodic boundary conditions and the initial condition $u(x, t=0) = e^{7(x-\pi)^2}$. The exact solution is  $u(x, t) = e^{7(x-t-\pi)^2}$. The parameter $c$ is set to 1. The Fourier spectral method is used for the spatial differencing with 70 modes.
\\
\hline
3. Nonlinear PDE : Viscous Burgers equation (`burgers')

  This is a nonlinear PDE given by
 \begin{align}
     u_{t} +uu_{x} = \nu u_{xx}, \quad x \in (0,2\pi), \quad t \in (0,2)
 \end{align}

 This PDE is solved with periodic boundary conditions and the initial condition $u(x, t=0) = 2 \mu \sin x/(1.5+\cos x)
$. The exact solution is  $u(x, t) = 2 \mu e^{-\mu t}\sin x/(1.5+e^{-\mu t}\cos x)
$. The parameter $\nu$ is set to 0.1. The Fourier spectral method is used for the spatial differencing with 70 modes.
\\
\hline
\end{tabularx}
\end{table}

 The error for all ODEs and PDEs apart from the Van Der Pol oscillator is calculated by comparing the numerical solution to the exact solution. For the Van Der Pol oscillator, the numerical solution using complex paths is compared to the numerical solution obtained by using the `RK44' 4th order method from Nodepy \cite{ketcheson2020nodepy} with $\Delta t = 10^{-6}$.

\subsection{Higher order integrators tailored to specific problems}

For specific problems, it is possible to derive complex time steppers that are uniquely accurate with fewer steps. For example, we can derive a 2-step complex Euler method with 3rd order accuracy for the following nonlinear problem,

\begin{align}
    \label{2step3}
    y' = -y^2 \quad y(t_0) = y_0.
\end{align}

The true solution at time  $t_0 + \Delta t$ can be approximated to third order accuracy using Eq \eqref{nonlintay} to obtain
\begin{align}
    \label{2step3soln}
    y(t_0 \Delta t) = \frac{y_0}{1+ y_0 \Delta t} = y_0 - y_0^2 \Delta t  + y_0^3 \Delta t^2   - y_0^4 \Delta t^3 + O(\Delta t^4).
\end{align}
Note that one does not need the true solution to obtain Eq \eqref{2step3soln} as this expansion can be obtained from the right hand side of the differential equation.

Now, we apply the following two-step complex  method to integrate from $y_0$ at time $t = t_0 $ to $y_2$ at time $t = t_0 +\Delta t$ using forward Euler steps.

\begin{align}
    \label{2step3comp}
    y_0 &= y_0 \nonumber\\
    y_1 &= y_0 + w_1 \Delta t y_0^2 \nonumber\\
    y_2 &= y_1 + w_2 \Delta t y_1^2
\end{align}

where $w_1 \Delta t, w_2 \Delta t$ are complex time steps.
Writing $y_2$ in terms of the initial condition, we get

\begin{align}
    \label{2step3compend}
 y_2 = y_0 + (w_1 + w_2) y_0^2 \Delta t + 2w_1 w_2 y_0^3 \Delta t^2 +  w_1^2 w_2 y_0^4 \Delta t^3.
\end{align}

This 2 step complex Euler method would have 3rd order accuracy if the coefficients in Eq. \ref{2step3soln} and \ref{2step3compend} match. However, there are no values $w_1, w_2$ that satisfy these conditions. So, we assume the case in which the true solution to \eqref{2step3} is real at real time points (valid if the initial condition is real). For this case, we can let the error be purely imaginary beyond order 1 and use the trick of taking the real part of the numerically calculated values every time we complete a complex path and return to the real line. This relaxed constraint produces the following a nonlinear system of equations that can be solved,

\begin{align}
    \label{nonlinsys2step3}
    w_1 +w_2 = 1 \nonumber\\
     \textbf{Re}(w_1w_2) = \frac{1}{2} \nonumber\\
       \textbf{Re}(w_1^2 w_2) = 1.
\end{align}

One of the many solutions to this system is
 \begin{align}
     w_1 = 1 -\frac{1}{\sqrt{2}} i  \quad w_2 = \frac{1}{\sqrt{2}} i.
 \end{align}

Applying this new 2-step time stepper to the nonlinear problem, it indeed has 3rd order accuracy (Fig \ref{2step3fig}). Similar 2-step paths are derived for $\dot{y} = f(t)$ and $\dot{y} = -e^y$ and they also show expected 3rd order accuracy in Fig. \ref{2step3fig}.

  \begin{figure}
\begin{centering}
\includegraphics[width=5in]{2step3.pdf}
\caption{Two step complex method Euler methods that have third order accuracy for two specific problems.}
 \label{2step3fig}
\end{centering}

\end{figure}

In the process of constructing this integrator, we needed to plug in the right hand side of the differential equation into Eq. \ref{nonlintay} to derive the expansion. This may be an intensive symbolic task for more complicated problems. However, the right complex steps  could be of `learned' for specific and difficult problems through approaches similar to \cite{guo2021personalized} where a machine learning approach is used to obtain Runge-Kutta methods tailored to particular ODE families.

In this section, we have shown the process of constructing higher order paths for explicit Runge-Kutta type integrators. Later, in Section \ref{implicitsection}, we extend it to implicit methods. In the next section, we discuss the benefits and costs when compared to equivalent classical Runge-Kutta methods.

\section{Operation counts and low storage benefits} \label{section:cost}
 Now, we discuss a potential drawback of the complex timestepping schemes, the increased computational cost for performing complex operations, and a potential benefit, the low storage cost compared to the Runge-Kutta methods. In Sec. \ref{sub:schrodinger}, we demonstrate that low storage  cost is the main benefit for complex-valued systems.

 Consider the forward Euler method applied to the differential equation $\dot{y} = f(t,y)$, yielding the step $ y_{i+1} = y_i + \Delta t f(t,y)$. If the timestepping is done along the real line, the computational cost is one real multiplication, one real addition, and the operation cost of one real function evaluation. If the timestepping is done along a path in the complex plane, the computational cost is one complex multiplication, one complex addition and the operation cost of one complex function evaluation. A complex multiplication costs 4M + 2A or 3M+5A (Gauss's trick), where M and A are the costs of real multiplication and addition. The storage also needs to be doubled to store a complex-valued vector compared to the real-valued vector.

Consider taking a complex timestep $w\Delta t$ with  the  ordinary differential equation $\dot{y}= \lambda y$. We can express it as  the following system.
\begin{align} \label{cost}
    \bcm y_r  \\y_i\ecm^{k+1} = \bcm y_r  \\y_i\ecm^{k} + \bcm (\lambda w)_r \Delta t & - (\lambda w)_i \Delta t \\ (\lambda w)_i \Delta t & (\lambda w)_r \Delta t\ecm \bcm y_r  \\y_i\ecm^{k}
\end{align}

where $w$ represents the complex time step and the subscripts $r$ and $i$ represent the real and imaginary parts. This results in 4M+4A cost of operations as opposed to M+A operations for purely real time steps. Based on Eq \eqref{cost}, the complex time stepping process shows promise for parallelization, which could reduce computational time by up to half.

Similarly, if we integrate the system of N equations $\dot{y} = A y$ with complex time steps, it becomes a system of $2N$ equations. In that scenario, the dominant cost is the matrix-vector multiplication ($Ay$) which takes $N^2$ operations with real time steps and $4N^2$ operations with complex time steps. For a k-step Complex Euler method,there are k matrix-vector multiplications and  the operation count is $4kN^2$ compared to $kN^2$ for a k stage  Runge-Kutta Method.

For nonlinear systems, the dominant cost comes from function evaluations. Functions can be designed to minimize the computational cost, but the real equivalent of any complex-valued function can always be calculated using fewer operations. So, the complex time stepper will always be more expensive than an explicit Runge-Kutta method if the number of function evaluations is the same. In the next section, we explore how complex time  integrators can require less function evaluations than the classical Runge-Kutta methods of the same order by breaking the Runge-Kutta order barrier \cite{butcher2009fifth}. In that case, the complex integrators may end being the less expensive option.

Another potential advantage, is that the storage requirement for complex stepping is a vector of size 2N while general Runge Kutta scheme store the intermediate stages. For example, a 5th order complex timestepping scheme needs to storage a vector of size $2N$ at all times while a classically implemented 5th order Runge-Kutta method requires memory for vector of size $5N$. However, there are low-storage implementations of Runge-Kutta  schemes that use fewer registers \cite{carpenter1994fourth,KENNEDY2000177, ketcheson2008highly}.  Storage is not much of an issue for typical odes and simple systems but it becomes quite relevant in various practical settings like in computational fluid dynamics \cite{KENNEDY2000177} where partial differential equations like the Navier Stokes equations are solved in 3 dimensions at high resolution.

\subsection{The linear Schrodinger equation : A clear computational advantage for the 3-step complex Euler method}\label{sub:schrodinger}

In this subsection, we demonstrate an example case where our complex methods have a clear benefit over existing methods. We compare the 3-step complex Euler method to a standard RK-3 method (Ralston's method) for a specific problem. We choose to examine the linear Schrodinger equation because it is a partial differential equation with complex values, so the cost of complex operations is the same in both methods. We show that methods are of the same order, but because 3-step complex Euler has lower storage requirements, it is advantageous.

The linear Schrodinger equation is given by
 \begin{align}
     u_{t} = i u_{xx}, \quad x \in (0,2\pi).
 \end{align}

We solve the equation from $t = 0$ to $t= 3$ on the domain $x \in (0, 2 \pi)$ with periodic boundary conditions. The initial condition is given by $u(t=0)=e^{ix}$ and the Fourier spectral method is used for the spatial differencing. The exact solution is given by $u(t) = e^{i(x-t)}$.

\begin{figure}
\begin{centering}
\includegraphics[width=5.1in]{schrodinger_1.pdf}
\caption{The linear Schrodinger is solved using both Runge-Kutta 3 (Ralston's method) and the complex 3-step method. Both methods result in nearly identical error ($10^{-9}$ using the infinity norm). }
\end{centering}
\label{fig:schro_1}
\end{figure}

\begin{figure}
\begin{centering}
\includegraphics[width=5.1in]{schrodinger_2.pdf}
\caption{Runge-Kutta 3 (Ralston's method) and the complex 3-step method require the same the number of function calls and result in nearly identical error when computing the linear Schrodinger equation.}
\end{centering}
\label{fig:schro_2}
\end{figure}

Figs \ref{fig:schro_1} and \ref{fig:schro_2} show that the two methods have near identical error and require equal number of function evaluations when integrating the linear Schrodinger equation. The 3-step complex method has one advantage over the Runge-Kutta-3 method:  Since the calculating the next step only requires the numerical solution at the current step, the required storage is the size of the initial condition vector. The Runge-Kutta-3 method requires thrice the storage to store the 3 stages during each function evaluation in its basic implementation.

\section{Breaking the order barrier for explicit Runge-Kutta methods with complex coefficients}\label{subsec:orderbarrier}

One can construct an $s$ stage explicit Runge-Kutta method with order $p$ where $ s =  p$ up to order $p =4$. But for order 5 and beyond, the number of stages needs to be more than the order desired \cite{butcher2009fifth}. This is known as the order barrier for Runge-Kutta methods \cite{Butcher:2007}.

Specifically for the 5th order Runge-Kutta-5 method, 6 stages are required and hence 6 function evaluations. This is because for 5th order accuracy, there are 17 order conditions and the 5 stages only provide 15 free parameters to satisfy them.

Can the order barrier be broken with complex time stepping? If we take $n$ complex Euler steps and use the trick of taking the real part and keeping the $>1$ order error purely imaginary, we have $2n-1$ free parameters ($n$ complex variables that need to return to the real line). $2n-1$ free parameters are insufficient even for 4th order accuracy because there are 8 order conditions and only 7 free parameters for $n  =4$.

Complex stepping can be applied to numerical methods apart from Forward Euler as well. What if we allowed some coefficients in a 5-stage Runge Kutta step to be complex variables and constrained the error to be purely imaginary? That easily increases the number of free parameters from 15 to 17 or higher in order to solve the 17 order conditions!

Another approach to break the $p =5$ order barrier is to take two complex Runge-Kutta steps: one Runge-Kutta-2 step and then one Runge-Kutta-3 step. If we require that the two complex Runge-Kutta steps come back to the real line, the total number of free parameters is 10 (3 free parameters for Runge-Kutta-2, 6 free parameters for Runge-Kutta-3 and the two time steps sizes with the constraint that they add up to $\Delta t$ (purely real)). If all 10 of those free parameters are complex valued and the $>1$ order error is purely imaginary, we get 20 free parameters which are sufficient to solve the 17 order conditions with 5th order accuracy. Using this method, we can achieve 5th order accuracy with only 5 function evaluations, breaking the order barrier. As these are complex function evaluations, the cost is higher than the real equivalent, but there are problems and architectures for which this is likely not an issue as discussed in Sec. \ref{section:cost}.

As a proof of concept, we find a 5th order accurate complex Runge-Kutta-2/Runge-Kutta-3 method for the special case of scalar autonomous ODEs.  We choose this particular case because it is more tractable than the 17 order conditions for the general case, which require a more complicated derivation involving a  rooted tree approach \cite{butcher2009fifth}. Even with the scalar autonomous case, there are 12 order conditions to satisfy (which cannot be done with only 10 free parameters of the purely real version of the Runge-Kutta-2/Runge-Kutta-3 method). Here, we show that the complex Runge-Kutta-2/Runge-Kutta-3 method returns a 5th order accurate numerical solution for scalar autonomous ODEs with 5 function evaluations, something that cannot be done with its real equivalent.

 \begin{figure}
\begin{centering}
\includegraphics[width=5in]{order_barrier.pdf}
\caption{The complex two step Runge-Kutta-2/Runge-Kutta-3 method with complex Runge-Kutta coefficients has 5th order accuracy for autonomous scalar ODEs. }
 \label{order_barrier}
\end{centering}
\end{figure}

The following equations describe our two step complex method. First an Runge-Kutta-2 step of size $(b_{11}+b_{12}) \Delta t$ goes from $y_n$ to $y_m$ and then an Runge-Kutta-3 step of size $(b_{21}+b_{22}+b_{23}) \Delta t$ to goes from $y_m$ to $y_{n+1}$. The steps and stages are
\begin{align}
    \label{orderbarrier}
    y_n &= y_n\\
    k_{11} &= f(y_n)\\
    k_{12} &= f(y_n + a_{121}\Delta t k_{11})\\
    y_m &= y_n + b_{11} \Delta t k_{11}+ b_{12} \Delta t k_{132} \\
    k_{21} &= f(y_m)\\
     k_{22} &= f(y_m + a_{221}\Delta t k_{21})\\
     k_{23} &= f(y_m + a_{231}\Delta t k_{21} + a_{232}\Delta t k_{22})\\
    y_{n+1}  &= \textbf{Re}(y_m +b_{21}  \Delta t k_{21}+b_{22}  \Delta t k_{22}+b_{23}  \Delta t k_{23}).
\end{align}
 Ensuring the method returns to the real line at an effective time step of $\Delta t$ introduces the constraint $b_{11}+b_{12} + b_{21}+b_{22}+ b_{23} = 1$.

 With these 9 complex variables, we solve the 12 order conditions and the 1 constraint guaranteeing that the time-stepper returns to the real line. The resulting scheme provides 5th order accuracy for scalar autonomous ODEs with only 5 function evaluations (Fig. \ref{order_barrier}).

\section{Expanded regions of stability with complex time-steps} \label{stabilitysection}

In the previous sections, we designed paths in the complex plane to achieve higher order. In this section, we leverage complex paths in an alternative way and expand the time steppers's region of absolute stability. First, we discuss the linear stability regions of the complex integrators developed in Sec. \ref{subsec:linearFEpaths}. Next, we show how these regions can be expanded by designing the complex steps to optimize a desired stability polynomial.
Expanded stability regions have been studied in a similar manner previously in the context of Runge-Kutta methods where the order of accuracy for the Runge-Kutta method is sacrificed to obtain a larger stability region \cite{riha1972optimal, lawson1966order,abdulle2000roots, bogatyrev2004effective, ketcheson2013optimal, parsani2013optimized, kubatko2014optimal}. In most previous works, a new Runge-Kutta integrator (different integration method) was derived that had a stability function/polynomial corresponding to the desired stability region. Here, we keep the same integration method and traverse a path in the complex plane to obtain the desired stability region (different integration domain). We illustrate this approach using the forward Euler method. We also show that complex stepping may enable even larger regions designed for specific problems.

Consider the linear equation $\dot{y} = \lambda y$ studied in Sec. \ref{subsec:linearFEpaths}. Most integrators, including those discussed here, convert the linear equation into the difference equation $y(t+\Delta t) = \Phi(\lambda \Delta t)y(t)$ where $\Phi(z)$ is the stability function of the integrator \cite{leveque2007finite}. The region $z \in \mathbb{C}$ where $|\Phi(z)|\leq 1$ is the region of absolute stability. For a complex $n$-step $n$th order method, the stability function is given by the first $n +1$ terms in the Taylor  series of $e^{\Delta t}$. For the 3 step 3rd order method, the stability function is

\begin{align}
\Phi(z) = 1 + z + \frac{z^2}{2} + \frac{z^3}{3!}.
\end{align}

The regions of absolute stability for the 1 step 1st order, 2 step 2nd order and 3 step 3rd order methods (Fig \ref{4_1}) are the same as those for the n-stage Runge-Kutta because they have the same stability functions \cite{leveque2007finite}. Note that $\Delta t$ in all these methods refers to the step size separating the beginning and end points of the complex path and hence a real time step. Since the 1-step, 2-step and 3-step methods require a different number of steps and hence a different number of function evaluations to take a net step of size $\Delta t$, a standard metric for comparison among methods is the effective step size $\Delta t_{eff}  = \Delta t/n$ where $n$ is the number of steps in the complex integrator case or the number of stages in the case of Runge Kutta methods \cite{ketcheson2013optimal}.

 \begin{figure}
\begin{centering}
\includegraphics[width=5in]{sec4_1_ink}
\caption{The 1-step first order, 2-step 2nd order and 3-step 3rd order complex Euler methods share the same stability regions as the 1,2 and 3 stage Runge-Kutta methods (left). Corresponding complex time-stepping paths are given on right.}
 \label{4_1}
\end{centering}
\end{figure}

We would like to expand the regions in Fig \ref{4_1} by optimizing the stability function. If we relax our requirement for a 3-step integrator to be 3rd order accurate, and require only 2nd order accuracy, its stability function would be given by

\begin{align}
\Phi(z) = 1 + z + \frac{z^2}{2} + k z^3.
\end{align}

The coefficient $k$ in the above expression is the same as the coefficient in front of the $\lambda^3 \Delta t^3$ in Eq \eqref{eq:linearcoeff}: $w_1w_2w_3$. We can choose the complex steps $w_1, w_2$ and $w_3$ such that they satisfy the first and second order conditions ($w_1 + w_2 + w_3 = 1, w_1 w_2 +w_2w_3 +w_1w_3 = 1/2$)and make $w_1 w_2 w_3$ equal to our desired value of $k$. The optimal $k$ value is the one that would allow us to take the largest stable time step. For problems with purely real and imaginary eigenvalues, the optimal $k$ values and the associated optimal stability functions/polynomials have been long-studied \cite{riha1972optimal,bogatyrev2004effective,abdulle2000roots}. In the past decade, Ketcheson and co-authors \cite{parsani2013optimized, kubatko2014optimal, ketcheson2013optimal} have explored optimal stability functions in the context of $\dot{y} = L y$ where $L$ is a matrix with quite arbitrary spectra. In \cite{ketcheson2013optimal}, Ketcheson and Ahmadia  describe an optimization algorithm to obtain the optimal stability polynomial that allows the maximal time step depending on the spectra of $L$. This algorithm (RKOpt \cite{ketcheson2020rk}) is publicly available through Nodepy \cite{ketcheson2020nodepy}.

By choosing the value $k$ that results in the largest permissible time step along the negative real eigenvalue line, we achieve a 3-step 2nd order method (green) with expanded stability along the negative real axis compared to the 3-step 3rd order method (red Fig \ref{4_2}). Requiring only 1st order accuracy for the 3-step method can nearly triple the size of the stability region along the negative real axis (gold) and surprisingly requires only real time-steps (Fig \ref{4_2}).
  \begin{figure}
\begin{centering}
\includegraphics[width=5in]{sec4_2_ink.pdf}
\caption{By sacrificing accuracy, one can expand the stability region (left) using complex time steps, as demonstrated by these optimized 3-step complex Euler paths (right) with 3rd, 2nd, and 1st order accuracy.}
\label{4_2}
\end{centering}
\end{figure}

 To our knowledge, our work is the first to use complex time steps to create stability polynomials, enabling us to expand the stability region. Next, we will demonstrate a potential path for even larger stability regions via complex stepping.

The use of complex time steps in an integrator allows to expand the optimal stability polynomials beyond those with real coefficients. Consider the challenge of finding a 3-step 2nd order integrator with an optimal stability region that encompasses the largest possible timestep along the eigenvalue $\lambda = -1 -2 i$ (red dots Fig \ref{4_3equation}). The red dots further away from the origin correspond to larger $\Delta t$ values along the eigenvalue. We want to find the optimal stability polynomial/function $\Phi(\lambda \Delta t)$ that permits the largest $\Delta t$ for a 3 stage/step 2nd order method in this scenario.

The optimal stability polynomial returned by RKOpt\cite{ketcheson2020rk} (magenta Fig. \ref{4_3equation}) is

\begin{align}
    \Phi(z) = 1 + z + \frac{1}{2}z^2 + 0.1134z^3.
\end{align}
Through trial and error, we find that an even better stability polynomial (brown Fig. \ref{4_3equation}) given by
\begin{align}
    \Phi(z) = 1 + z + \frac{1}{2}z^2 + 0.1134z^3- 0.06 i z^3
\end{align}
The complex coefficient allows for a larger stable time step size for a problem with this eigenvalue. A trade-off is that the optimal region is no-longer symmetric. So such regions would be primarily useful for complex problems, where complex eigenvalues appear outside conjugate pair. Alternatively, better stability region designs may be possible using more sophisticated optimization methodology.

  \begin{figure}
\begin{centering}
\includegraphics[width=5in]{sec4_3_ink.pdf}
\caption{Using complex time steps produces stability polynomials with complex coefficients. The resulting stability regions (left) can expand further in a desired direction than those using purely real coefficients. Corresponding paths on right. }
\label{4_3equation}
\end{centering}

\end{figure}

This section has been an exploration of the paths in the complex plane on which an integrator has improved linear stability properties.  In the next section, we consider nonlinear stability properties particularly strong stability properties. Later, in Section \ref{implicitsection}, we discuss the derivation of implicit complex  methods with properties like L-stability.

\section{Remarks on Strong Stability Preserving  (SSP) properties}
\label{section:ssp}

Time integrators with SSP properties are desirable because they extend they provide guarantees of stability for nonlinear problems and non-smooth solutions. In this section, we highlight the need for new analysis for deriving practical SSP conditions for complex timesteppers and we show numerically that complex timestepping can result in larger SSP time steps.

Consider the following differential equation system,
\[u_t = F(u).\]

If one use the forward Euler method is used to integrate the above system, the system would be said to satisfy a SSP property if
\[ |u_{n+1}| = |u_n + \Delta t F(u_n)| \leq |u_n| \quad  \text{for} \quad 0 \leq \Delta t \leq \Delta t_{FE} \]

where the maximal SSP time step is given by

\[|\Delta t_{FE}| = \min_{n}    \frac{2}{\bigg|\frac{ F(u_n)}{u_n}\bigg|}.\]

Much effort has focused on obtaining optimal SSP Runge-Kutta methods in various contexts \cite{gottlieb2001strong, ketcheson2008highly}. The general argument used to show SSP is as follows; if forward Euler is SSP for some $\Delta t \leq \Delta t_{FE}$, these optimal SSP Runge-Kutta methods can be expressed as a convex combination of the forward Euler steps and guaranteed to be SSP for some $\Delta t \leq C \Delta t_{FE}$. This $C$ is known as the SSP coefficient.

Following this line of argument, consider our $k$ step general complex Euler time stepper expressed as

\begin{align*}
    u^{0} &= u_n \\
     u^{i} &= u^{i-1} + w_i \Delta t F(u^{i-1}),  i = 1, \hdots, k \\
     u_{n+1} &= u^k.
\end{align*}

Here, $u_n, u_{n+1}$ represent the solution at $n$ and $n+1$ time points on the real line. The $u^i$ represent the solution at the complex time points in between and the $w_i \Delta t$ are the complex time steps. Unlike the Runge-Kutta methods in their Shu-Osher form, finding an SSP coefficient using convex combinations of forward Euler is less straightforward because of the complex nature of $w_i$ and $F(u^{i-1})$.

A strict condition for an SSP complex time integrator can be derived  if we require
\[ |u^{i} |= |u^{i-1} + w_i \Delta t F(u^{i-1})| \leq |u^{i-1}|, \quad   i = 1, \hdots, k .\]

This results in a very restrictive time step.

\[\Delta t \leq  \min_i \left(-2  \frac{\textbf{Re}\bigg(\frac{w_i  F(u^{i-1})}{u^{i-1}}\bigg)}{          |w_i|^2\bigg|\frac{ F(u^{i-1})}{u^{i-1}}\bigg|^2}\right). \]

 Notably, the lower bound on the largest possible SSP timestep can be as small as zero if $\textbf{Re}\bigg(\frac{w_i  F(u^{i-1})}{u^{i-1}}\bigg)$ vanishes. This is too strict of a condition to obtain a practical SSP timestep. We can obtain a larger timestep with a looser SSP condition if we require that only the solutions at the real time points at the end of the path in the complex plane satisfy the SSP property $|u_{n+1} | \leq |u_n|$. For the two step second order complex time integrator which takes the complex steps $w$ and $\bar{w}$ at each real time point, where $w, \bar{w}$ are $\frac{1}{2}+\frac{1}{2}i$ and $\frac{1}{2}-\frac{1}{2}i$, , this translates to the following condition.

\[ |u_n + w \Delta t F(u_n) + \bar{w}F(u_n + w \Delta t F(u_n))| \leq |u_n|\]

We can also take the  $\bar{w}$ step first and then the  $w$ step before returning to the real line. Taking different paths with the same number of steps and the same order results in different SSP conditions.
 However, we have not yet been able to obtain a clean expression for an upper bound on the SSP timestep using this approach.

Using numerical experiments, we find the complex timestepping methods could allow for a larger SSP stable timestep. We demonstrate for the simple case,

 \begin{align}
     \dot{y} = f(y) = -ye^{-y}.
 \end{align}

 We consider three 2nd order timestepping methods.

 \begin{itemize}
     \item The midpoint method:

     \begin{align}
         u_{n+1} = u_n + \Delta t_n f(u_n+ \Delta t_n/2 f(u_n))
     \end{align}

       \item The optimal SSP Runge-Kutta 2 method: This is the second order RK method with the optimal SSP coefficient of 1. \cite{gottlieb2001strong}

     \begin{align}
        u_{m} &= u_n + \Delta t_n f(u_n)\nonumber\\
         u_{n+1} &= \frac{u_n}{2} + \frac{u_m}{2}+ \frac{1}{2}\Delta t_n f(u_m)
     \end{align}

    \item The complex 2-step method: We assume our initial condition is real. So, we make sure to take the real part of the solution at the end of the two steps.

     \begin{align}
        u_{m} &= u_n + w_1\Delta t_n f(u_n)\nonumber\\
         u_{n+1} &= \textbf{Re}(u_m + w_2\Delta t_n f(u_m))
     \end{align}
 \end{itemize}

 We numerically compute the largest timestep $\Delta t_n$ for each method for various $u_n$ values such that for $\Delta t \leq \Delta t_n$,  the SSP condition ($|u_{n+1}| \leq |u_n|$) is satisfied. Different methods permit the largest time-step for different sizes of $u_n$. However, for most values, the complex 2-step method enables comparable maximal time-steps and for large values of $u_n$, the complex 2-step method appears to enable larger maximum time-steps (Fig. \ref{ssp}). While we have not yet been able to prove SSP properties for complex methods, these numerical experiments show it may have desirable properties.

       \begin{figure}[H]
\begin{centering}
\includegraphics[width=5in]{SSP.pdf}
\caption{For various values of $u_n$, the complex 2-step method allows us to take a larger SSP timestep.}
 \label{ssp}
\end{centering}
\end{figure}

\section{Dealing with stiff systems: Implicit methods}\label{implicitsection}

When dealing with stiff systems, expanding the stability region of the integrator is rarely sufficient and implicit methods with properties like L-stability are required. In this section, we derive a higher order complex path for an implicit method: The implicit midpoint method. First we study linear ODEs (Sec. \ref{stiff_linear}) and then we extend to general, nonlinear, non-autonomous differential equations (Sec. \ref{stiff_nonlinear}).

\subsection{Linear differential equations}\label{stiff_linear}

Consider the linear equation $\dot{y} = \lambda y$. Applying the implicit midpoint method to this equation produces the following difference equation,

\begin{align}
    y_{n+1} = y_n + \frac{\lambda \Delta t}{2} (y_n + y_{n+1})
\end{align}
which can also be written as
\begin{align}
    y_{n+1} = y_n\frac{1+\lambda \Delta t/2}{1-\lambda \Delta t/2}.
\end{align}

Now, suppose we take two complex steps $w_1 \Delta t$ and $w_2 \Delta t$ to go from $y_n$ to $y_{n+1}$. Our numerical approximation at $y_{n+1}$ is

\begin{align}
\label{impcomplex}
    y_{n+1} = y_n\frac{(1+\lambda w_1 \Delta t/2)(1+\lambda w_2 \Delta t/2)}{(1-\lambda w_1 \Delta t/2)(1-\lambda w_2 \Delta t/2)}.
\end{align}

If $y(t) = y_n$, the true solution at time $t+\Delta t$ would be $y_ne^{ \lambda \Delta t}$. This solution can be represented to $4$th order accuracy by the following rational function expansion of the exponential function, known as the (2,2) Pade approximate,

\begin{align}
\label{padeimp}
    y_{t + \Delta t} = y_n\frac{1+\lambda \Delta t/2 +\lambda^2 \Delta t^2/12}{1-\lambda \Delta t/2 +\lambda^2 \Delta t^2/12} + O(\Delta t^5).
\end{align}

Setting the right hand sides of \eqref{impcomplex} and \eqref{padeimp} equal to each other, gives us the 2-step path in the complex plane on which the implicit midpoint method has $4$th order accuracy.

By solving
\begin{align}
    w_1 + w_2 = 1 \quad, \quad w_1w_2 = \frac{1}{3},
\end{align}
we find that the two 2-step 4th order paths are $(\frac{1}{2} + i \frac{1}{2 \sqrt{3}}, \frac{1}{2} - i \frac{1}{2 \sqrt{3}})$ and  $(\frac{1}{2} - i \frac{1}{2 \sqrt{3}}, \frac{1}{2} + i \frac{1}{2 \sqrt{3}})$.

Equations \eqref{impcomplex} and \eqref{padeimp}, provide a clear connection between the optimal k-step complex path for the implicit midpoint method and the (k,k) Pade approximant.
We can derive the optimal k-step complex path for the Backward Euler method in a similar way, by setting it equal to the (0,k) Pade approximate. This results in the same paths derived in Subsection \ref{subsec:linearFEpaths} for the forward Euler method.

We test out our newfound implicit paths on the heat equation with Dirichlet boundary conditions.
\begin{align}
    u_t = u_{xx}, \quad  x \in (0,1)
\end{align}

We choose the initial condition $u(x,0) = \sin(\pi x)$, which has the exact solution  $u(x,t) = e^{-\pi^2 t}\sin(\pi x)$. The spatial discretization is performed using 4th order finite differencing, periodic boundary conditions and a grid size of 10000 cells. As expected, the 2-step complex implicit midpoint method has 3rd order accuracy and the complex 3-step backward Euler method has 4th order accuracy (Fig. \ref{implicit_linear}).
      \begin{figure}
\begin{centering}
\includegraphics[width=5in]{implicit_heat_ink.pdf}
\caption{The 2-step implicit midpoint method has 4th order accuracy while the 3-step Backward Euler has 3rd order accuracy for the heat equation.}
 \label{implicit_linear}
\end{centering}
\end{figure}

 \subsection{General nonlinear non-autonomous differential equations} \label{stiff_nonlinear}

 In the previous subsection, we found that the path $(\frac{1}{2} + i \frac{1}{2 \sqrt{3}}, \frac{1}{2} - i \frac{1}{2 \sqrt{3}})$  allows the implicit midpoint method to be 4th order accurate when solving linear differential equations. Does the expected 4th order accuracy of the 2-step complex path for the implicit midpoint method derived in the previous subsection hold for general nonlinear differential equations?  Numerically, we confirm this to be true, provided we use the trick of taking the real part of the numerically calculated values at the end of the complex path. To prove 4th order accuracy analytically, we analyze the midpoint-method following a similar line of reasoning as our previous nonlinear analysis (Equations  \eqref{taylornonlinexp} - \eqref{3comptay}). Suppose that we take two implicit steps $w_1 \Delta t$ and  $w_2 \Delta t$ from $y_n$ at $t_n$ to $y_{n+1}$ at at $t + \Delta t $ with $y_{m}$  being the intermediate solution at at $t+ w_1 \Delta t$.

\begin{align}
    \label{impnon1}
    y_n &= y(t_n)\\
    \label{impnon2}
    y_m &= y_n + w_1 \Delta t f(t_n+w_1 \Delta t /2,  (y_n +y_m)/2) \\
    \label{impnon3}
    y_{n+1}  &= y_m + w_2 \Delta t f(t_n+w_1 \Delta t+ w_2 \Delta t /2,  (y_m +y_{n+1})/2)
\end{align}

Unlike the previous analysis, equations \eqref{impnon2} and \eqref{impnon3} are implicit equations and cannot be directly Taylor expanded around $f(t,y_n)$. To overcome this difficulty and calculate the desired $w_1$ and $w_2$ step sizes, we replace the two implicit equations with explicit versions that are accurate to the original implicit equations to at least 4th order. The explicit versions are found by recursively substituting the implicit equation into itself in an iterative process similar to Picard Iteration. For example, if you have an
implicit equation given by
\begin{align}
    y = y_0 + \Delta t g(y)
\end{align}
one can recursively apply it twice to obtain the following explicit equation, which is accurate to the original implicit equation to the 3rd order.

\begin{align}
    y = y_0 + \Delta t g(y_0 + \Delta t g(y_0 + \Delta t g(y_0)))
\end{align}

To obtain the 4th order accurate explicit approximation of equations \eqref{impnon2} and \eqref{impnon3}, they need to be recursively applied at least 3 times. Note that the explicit equations are 4th order accurate approximations to the original implicit difference equations and not the original differential equation.

At the end of this process,, we substitute $(w_1, w_2) = (\frac{1}{2} + i \frac{1}{2 \sqrt{3}}, \frac{1}{2} - i \frac{1}{2 \sqrt{3}})$ into the explicit approximation of equation \eqref{impnon3} and obtain

\begin{align}
\label{implicittay}
    y_{n+1} = &y_0 + \Delta t f(t_n, y_n) + \frac{1}{2} \Delta t^2 \bigg( f{\left(t,y \right)} \frac{\partial}{\partial y} f{\left(t,y \right)} + \frac{\partial}{\partial t} f{\left(t,y \right)}
\bigg)\at[\Bigg]{t=t_n, y = y_n}  \nonumber \\
    & + \frac{1}{3!} \Delta t^3 \bigg(f^2{\left(t,y \right)} \frac{\partial^{2}}{\partial y^{2}} f{\left(t,y \right)} + \left(\frac{\partial}{\partial y} f{\left(t,y \right)}\right)^{2} f{\left(t,y \right)}\nonumber \\ &+ 2f{\left(t,y \right)} \frac{\partial^{2}}{\partial y\partial t} f{\left(t,y \right)} + \frac{\partial}{\partial t} f{\left(t,y \right)} \frac{\partial}{\partial y} f{\left(t,y \right)} + \frac{\partial^{2}}{\partial t^{2}} f{\left(t,y \right)} \bigg)\at[\Bigg]{t=t_n, y = y_n} \nonumber\\
    &+ \frac{1}{4!}dt^{4} \bigg( (1+0.096225i)f^{3}{\left(t,y \right)} \frac{\partial^{3}}{\partial y^{3}} f{\left(t,y \right)}\nonumber\\& + (4-0.288675i) f^{2}{\left(t,y \right)} \frac{\partial}{\partial y} f{\left(t,y \right)} \frac{\partial^{2}}{\partial y^{2}} f{\left(t,y \right)}  + (3+0.288675i) f^{2}{\left(t,y \right)} \frac{\partial^{3}}{\partial y^{2}\partial t} f{\left(t,y \right)}\nonumber\\& + 3 f{\left(t,y \right)} \frac{\partial}{\partial t} f{\left(t,y \right)} \frac{\partial^{2}}{\partial y^{2}} f{\left(t,y \right)} + f{\left(t,y \right)} \left(\frac{\partial}{\partial y} f{\left(t,y \right)}\right)^{3} \nonumber\\ & + (5-0.57735i) f{\left(t,y \right)} \frac{\partial}{\partial y} f{\left(t,y \right)} \frac{\partial^{2}}{\partial y\partial t} f{\left(t,y \right)}  +  (3+0.288675i)  f{\left(t,y \right)} \frac{\partial^{3}}{\partial y\partial t^{2}} f{\left(t,y \right)} \nonumber\\ & + \frac{\partial}{\partial t} f{\left(t,y \right)} \left(\frac{\partial}{\partial y} f{\left(t,y \right)}\right)^{2} + 3 \frac{\partial}{\partial t} f{\left(t,y \right)} \frac{\partial^{2}}{\partial y\partial t} f{\left(t,y \right)}  +(1-0.096225i) \frac{\partial^{3}}{\partial t^{3}} f{\left(t,y \right)}  \nonumber\\ &+ (1-0.288675i)\frac{\partial^{2}}{\partial t^{2}} f{\left(t,y \right)} \frac{\partial}{\partial y} f{\left(t,y \right)}
    \bigg)\at[\Bigg]{t=t_n, y = y_n} + O(\Delta t^5).
\end{align}

We compare \eqref{implicittay} to the first 5 terms of Taylor expansion in equation \eqref{nonlintay}.The error at O($\Delta t^4$) is purely imaginary, which demonstrates we can get 4th order accuracy by extracting the real component of the solution at real time points (as long as the true solution is real at real time points).

Now, that we have proved our two-step implicit midpoint method to be 4th order accurate for nonlinear systems, we test it on the Van Der Pol oscillator,

\begin{align}
    \dot{y} = x - \frac{x^3}{3} -\frac{\dot{x}}{\mu}
\end{align}
which can be rewritten as the following system

\begin{align}
    \dot{x} &= y \nonumber\\
    \dot{y} &= \mu (1-x^2)y-x.
\end{align}

We set $\mu = 10$ and the initial condition as $(x,y) = (2,0)$. We achieve the expected 4rd order accuracy for the 2-step complex implicit Method (Fig. \ref{implicit_linear}).

      \begin{figure}
\begin{centering}
\includegraphics[width=5in]{implicit_vdp_ink.pdf}
\caption{The 2-step implicit midpoint method has 4th order accuracy while the 3-step Backward Euler has 3rd order accuracy for the Van Der Pol oscillator.}
 \label{implicit_nonlinear}
\end{centering}
\end{figure}

\section{Discussion}
This paper is an exploration of the benefits and trade offs of leaving the real line to perform numerical wi integration in the complex plane. Our first goal was to explain the process of developing a complex time-stepper and its potential benefits in an accessible manner. For both linear and nonlinear systems, we demonstrate how to derive paths in the complex plane that increase the order of accuracy of any Runge-Kutta type integrator. The derivation of these paths for implicit methods involves a more complicated iterative process and the implicit paths for linear problems show us a natural connection to Pade approximants. For real valued differential equations, existing Runge-Kutta methods are less costly since they do not require complex arithmetic. However, for complex-valued differential equations such as the Schrodinger equation, the computational cost is similar and the low storage structure of complex time steppers may be beneficial.

Complex time steppers enable the major advance of breaking the Runge-Kutta order barrier\cite{butcher2009fifth}. Using complex time steps and complex Runge-Kutta coefficients, we can increase the number of variables available to satisfy the ever increasing number of order conditions and use fewer function evaluations to achieve the same order accuracy. For example, a traditional Runge-Kutta method requires at least 11 function evaluations to obtain 8th order of accuracy and complex timesteppers could reduce the number of evaluations down to 8. Whether one would ever need an 8th order method is a different question, but breaking the order barrier has been a long-standing challenge in the computational world.

We also showed that complex time stepping can be used to expand stability regions, allowing larger effective time steps. Expanded stability regions have been designed using Runge-Kutta methods \cite{riha1972optimal, ketcheson2013optimal}, but complex coefficients in the optimal stability polynomial may support even larger stability regions in specific cases. The fact that you can increase the stability region by changing the integration domain rather than the integration method may also be appealing from an easy user implementation standpoint. Our preliminary results about nonlinear stability properties such as SSP, suggest that future exploration may unearth specific paths for specific methods with desirable stability properties.

We intend this paper to be introduction to and initiation of the search for better time integrators that take advantage of paths in the complex plane. We have mostly focused on creating integrators with increased accuracy and stability, but there are many other desirable properties. The complex plane offers an extra dimension on which any integrator can be improved, opening up a new realm of possibilities when designing time stepping methods.

\section{Acknowledgements}

We would like to thank Emil Constantinescu, Alvin Bayliss and David Chopp for extensive feedback and valuable suggestions.

\bibliographystyle{unsrt}

Numerical Analysis (math.NA); Complex Variables (math.CV)