Approximations for the Caputo Derivative (I)

Abstract: In this paper we construct approximations for the Caputo derivative of order $1-\alpha,2-\alpha,2$ and $3-\alpha$. The approximations have weights $0.5\left((k+1)^{-\alpha}-(k-1)^{-\alpha}\right)/\Gamma(1-\alpha)$ and $k^{-1-\alpha}/\Gamma(-\alpha)$, and the higher accuracy is achieved by modifying the initial and last weights using the expansion formulas for the left and right endpoints. The approximations are applied for computing the numerical solution of ordinary fractional differential equations. The properties of the weights of the approximations of order $2-\alpha$ are similar to the properties of the $L1$ approximation. In all experiments presented in the paper the accuracy of the numerical solutions using the approximation of order $2-\alpha$ which has weights $k^{-1-\alpha}/\Gamma(-\alpha)$ is higher than the accuracy of the numerical solutions using the $L1$ approximation for the Caputo derivative.

\section{Introduction}
The Gr\"unwald formula approximation and the $L1$ approximation of the Caputo derivative have been regularly used for numerical solution of fractional differential equations \cite{JinLazarovZhou2016,LinXu2007,Ma2014,Murio2008,Podlubny1999}. The Gr\"unwald formula approximation has weights $(-1)^k\binom{\alpha}{k}$ and accuracy $O(h)$. The $L1$ approximation \eqref{3_10} has order $2-\alpha$ and weights $\sigma_k^{(\alpha)}=\llll((k-1)^{1-\alpha}-2k^{1-\alpha}+(k+1)^{1-\alpha}\rrrr)/\Gamma(2-\alpha)$. The  weights of the $L1$ approximation are linear combinations of terms which have  power $1-\alpha$. In the present paper we construct approximations of the Caputo derivative whose weights consist of terms which have power $-\alpha$ and $-1-\alpha$. The accuracy of the numerical solution of order $2-\alpha$ is influenced by the coefficient of the term $y''(x)h^{2-\alpha}$ in the expansion of the approximation. In Table 1, Table 3 and Table 9 we compute the error and the order of the numerical solutions of the fractional relaxation equation which use approximations \eqref{3_10},\eqref{5_30} and \eqref{6_20} of the Caputo derivative. In all experiments the accuracy of the numerical solutions using approximation \eqref{5_30} is higher than the accuracy of the numerical solutions using approximations \eqref{3_10} and \eqref{6_20}.
The Caputo derivative  of order $\alpha$, where $0<\alpha<1$ is defined as
$$y^{(\alpha)}(x)=D^{\alpha} y(x)=\dddd{1}{\Gamma (1-\alpha)}\int_0^x \dfrac{y'(t)}{(x-t)^{\alpha}}d t.$$
When the first derivative of the function $y$ is a bounded function the Caputo derivative at the initial point $x=0$ is equal to zero. The exponential function has Caputo derivative
$D^\alpha e^{\lambda x}=\lambda x^{1-\alpha}E_{1,2-\alpha}(\lambda x)$
and
$$D^\alpha \cos (\lambda x)=\dddd{i \lambda}{2} x^{1-\alpha}
\llll(E_{1,2-\alpha}(i \lambda x)-E_{1,2-\alpha}(-i\lambda x)\rrrr),$$
where $E_{1,2-\alpha}( x)$ is the Mittag-Leffler function ($\beta=2-\alpha$)
$$E_{\alpha,\beta}(x)=\sum_{k=0}^\infty \dddd{x^k}{\Gamma(\alpha k+\beta)}.$$
The analytical solutions of ordinary and partial differential equations are often expressed with the Mittag-Leffler function. The ordinary fractional differential equation
\begin{equation*}
y^{(\alpha)}(x)+D y(x)=F(x),\; y(0)=y_0,
\end{equation*}
has exact solution
$$y(x)= y_0 E_{\alpha}(-D x^\alpha)+\int_{0}^{x}\xi^{\alpha-1} E_{\alpha,\alpha}\llll(-D \xi^\alpha\rrrr)F(x-\xi)d\xi.$$
The finite-difference schemes for numerical solution of ordinary and partial fractional differential equations use an approximation for the fractional derivative.  Let $h=x/n$ and $y_n=y(n h)$ be the value of the function $y$ at the point $x_n=n h$. The  $L1$ approximation of the Caputo derivative is constructed by dividing the interval $[0,x]$ to subintervals of equal length $h$ and approximating the first derivative on each subinterval using a second-order central difference approximation.
\begin{equation}\label{3_10}
y^{(\alpha)}_n  =\dfrac{1}{\Gamma(2-\alpha)h^\alpha}\sum_{k=0}^{n} \sigma_k^{(\alpha)} y_{n-k}+O\llll(h^{2-\alpha}\rrrr),
\end{equation}
where $\sigma_0^{(\alpha)}=1$,  $\sigma_n^{(\alpha)}=(n-1)^{1-a}-n^{1-a}$ and
$$\sigma_k^{(\alpha)}=(k+1)^{1-\alpha}-2k^{1-\alpha}+(k-1)^{1-\alpha}, \quad (k=2,\cdots,n-1).$$
  Approximation \eqref{3_10} has accuracy $O(h^{2-\alpha})$ when $y\in C^2[0,x]$ (\cite{LinXu2007}). 	 The numbers $\sigma_k^{(\alpha)}$ have properties
\begin{align}\label{3_20}
&\sigma_0^{(\alpha)}>0,\; \sigma_1^{(\alpha)}<\sigma_2^{(\alpha)}<\cdots<\sigma_k^{(\alpha)}<\cdots<\sigma_{n-1}^{(\alpha)}<0,\;\sigma_{n}^{(\alpha)}<0,\nonumber\\
&\sum_{k= 0}^n \sigma_k^{(\alpha)} = 0,\quad \sum_{k= 1}^n k \sigma_k^{(\alpha)} = -n^{1-\alpha},\\
&\sigma_k^{(\alpha)}= \dddd{C_1}{k^{1+\alpha}}+O\llll(\dddd{1}{k^{2+\alpha}} \rrrr),\quad \sigma_n^{(\alpha)}= \dddd{C_2}{n^\alpha}+O\llll(\dddd{1}{n^{1+\alpha}} \rrrr),\nonumber
\end{align}
where $C_1=\alpha(\alpha-1)$ and $C_2=\alpha-1$.
{ \renewcommand{\arraystretch}{1.1}
\begin{table}[h]
	\caption{Error and order of numerical solution $NS[1]$ of Equation I with $\alpha=0.25$, Equation II with $\alpha=0.5$ and Equation III with $\alpha=0.75$.}
	\small
	\centering
  \begin{tabular}{ l | c  c | c  c | c  c }
		\hline
		\hline
		\multirow{2}*{ $\quad \boldsymbol{h}$}  & \multicolumn{2}{c|}{{\bf Equation I}} & \multicolumn{2}{c|}{{\bf Equation II}}  & \multicolumn{2}{c}{{\bf Equation III}} \\
		\cline{2-7}
   & $Error$ & $Order$  & $Error$ & $Order$  & $Error$ & $Order$ \\
		\hline \hline
$0.003125$    & $0.0000466$          & $1.6970$  & $0.0000513$           & $1.4857$    & $0.0024184$  & $1.2442$       \\
$0.0015625$   & $0.0000143$          & $1.7071$  & $0.0000183$           & $1.4901$    & $0.0010191$  & $1.2468$       \\
$0.00078125$  & $4.3\times 10^{-6}$  & $1.7150$  & $6.5\times 10^{-6}$   & $1.4931$    & $0.0004290$  & $1.2482$        \\
$0.000390625$ & $1.3\times 10^{-6}$  & $1.7212$  & $2.3\times 10^{-6}$   & $1.4952$    & $0.0001805$  & $1.2490$        \\
\hline
  \end{tabular}
	\end{table}
	}

	 In section 4 we derive the numerical solution $NS[*]$ of the fractional relaxation equation which uses approximation \eqref{20_10} for the Caputo derivative.  In Table 1 we compute the error and the order of numerical solution $NS[1]$ of Equation I and $\alpha=0.25$, Equation II and $\alpha=0.5$ and Equation III with $\alpha=0.75$. In Theorem 9 and Theorem 10  we prove that the numerical solution of the fractional relaxation equation converges to the exact solution with accuracy $O\llll(h^{2-\alpha}\rrrr)$  when the coefficient $D$ is positive or $D$ is a bounded negative number and the approximation of the Caputo derivative has order $2-\alpha$ and satisfies  \eqref{3_20}. In section 2 and section 5 we obtain approximations \eqref{5_30} and \eqref{6_20} of the Caputo derivative. Approximations \eqref{3_10},\eqref{5_30} and \eqref{6_20} have order $2-\alpha$ and satisfy properties \eqref{3_20}. The asymptotic expansions of the  integral approximations for the fractional derivatives involve the values of the  {\it  Riemann zeta function} defined as
		\begin{equation*}
	\zeta(\alpha)=
	\displaystyle{\sum_{n=1}^\infty \dddd{1}{n^\alpha}}, \;(\alpha>1),
	\quad	\zeta(\alpha)=\displaystyle{\dfrac{1}{1-2^{1-\alpha}}\sum_{n=1}^\infty \dddd{(-1)^{n-1}}{n^\alpha}}, \;  (\alpha>0).
	\end{equation*}
	When $\alpha<0$, the values of the Riemann zeta function are computed from the the functional equation
$$\zeta(\alpha)=2^\alpha\pi^{\alpha-1}\sin \llll(\dddd{\pi \alpha}{2} \rrrr)\Gamma (1-\alpha)\zeta (1-\alpha).$$
Another important special function in fractional calculus is the {\it polylogarithm function} is defined as
$$Li_\alpha(x)=\sum_{n=1}^{\infty}\dddd{x^n}{n^\alpha}=x+\dddd{x^2}{2^\alpha}+\cdots+\dddd{x^n}{n^\alpha}+\cdots$$
The polylogarithm function satisfies $Li_\alpha(1)=\zeta(\alpha)$ and has  expansion
\begin{align}\label{4_10}
Li_\alpha(x)=\Gamma(1-\alpha)\llll(\ln \dddd{1}{x}\rrrr)^{\alpha-1}+\sum_{n=0}^{\infty}\dddd{\zeta(\alpha-n)}{n!}\llll(\ln x\rrrr)^n,
\end{align}
where $\alpha \neq 1,2,3,\cdots$ and $|\ln x|<2\pi$.
In \cite{Dimitrov2015}  we obtained the  second-order expansion formula of the $L1$ approximation
\begin{align}\label{4_15}
\dddd{1}{\Gamma(2-\alpha)h^\alpha}\sum_{k=0}^n \sigma_k^{(\alpha)} y(x-kh)=y^{(\alpha)}(x)+&\dddd{\zeta(\alpha-1)}{\Gamma(2-\alpha)}y''(x)h^{2-\alpha}+O\llll(h^2\rrrr),
\end{align}
and the second-order approximation of the Caputo derivative
\begin{equation}\label{4_20}
y^{(\alpha)}_n=\dddd{1}{\Gamma(2-\alpha)h^\alpha}\sum_{k=0}^n \delta_k^{(\alpha)} y_{n-k}+O\llll(h^{2}\rrrr),
\end{equation}
where $\delta_k^{(\alpha)}=\sigma_k^{(\alpha)}$ for $2\leq k\leq n$ and
$$\delta_0^{(\alpha)}=\sigma_0^{(\alpha)}-\zeta(\alpha-1),\; \delta_1^{(\alpha)}=\sigma_1^{(\alpha)}+2\zeta(\alpha-1),\; \delta_2^{(\alpha)}=\sigma_2^{(\alpha)}-\zeta(\alpha-1).$$
The numbers $\delta_k^{(\alpha)}$ satisfy
\begin{equation}\label{4_30}
\delta_0^{(\alpha)}>0,\delta_1^{(\alpha)}<0,\delta_2^{(\alpha)}>0, \delta_3^{(\alpha)}<\cdots<\delta_k^{(\alpha)}<\cdots<\delta_{n-1}^{(\alpha)}<0,\delta_{n}^{(\alpha)}<0.
\end{equation}

	The fractional integral of order $\alpha>0$ on the interval $[0,x]$ is defined as
$$I^\alpha y(x)=\dddd{1}{\Gamma(\alpha)}\int_0^x (x-t)^{\alpha-1} y(t) d t.
$$
 Denote by $ K^{\alpha} y(x)$ the fractional integral $  K^{\alpha} y(x)=\Gamma(\alpha) I^{\alpha} y(x)$. In \cite{Dimitrov2016} we use the Fourier transform method and the asymptotic expansion \eqref{4_10} of the polylogarithm function to derive the asymptotic formula for the Riemann sum approximation of the fractional integral $ K^{\alpha} y(x)$ at the right endpoint.
\begin{align}  h^{\alpha} \sum_{k=1}^{n-1} &\dddd{y(x-k h)}{k^{1-\alpha}}=\int_0^x \dddd{y(t)}{(x-t)^{1-\alpha}}dt +\sum_{k=0}^\infty (-1)^k\dddd{\zeta(1-\alpha-k)}{k!}y^{(k)}(x) h^{k+\alpha}\nonumber\\
&-\Gamma(\alpha)\sum_{k=0}^\infty \dddd{B_{k+1}}{(k+1)!}\llll( \sum_{m=0}^k (-1)^m\binom{k}{m}\dddd{x^{\alpha-m-1}}{\Gamma(\alpha-m)}y^{(m-k)}(0) \rrrr)h^{k+1}.\label{5_10}
\end{align}
The asymptotic expansion formula for the left endpoint of \eqref{5_10} is obtained from the Euler-Maclaurin formula for the function $z(t)=y(t)(x-t)^{\alpha-1}$.
In section 2, we use \eqref{5_10} to obtain the approximation of the Caputo derivative
\begin{align}\label{5_20}
\dddd{1}{2h^\alpha}\Bigg(y_n+&\dddd{y_{n-1}}{2^\alpha} +\sum_{k=2}^{n-2} y_{n-k}\llll(\dddd{1}{(k+1)^\alpha}-\dddd{1}{(k-1)^\alpha}  \rrrr)-\dddd{y_{1}}{(n-2)^\alpha} -\dddd{y_{0}}{(n-1)^\alpha} \Bigg)\nonumber\\
&=\Gamma(1-\alpha)y_n^{(\alpha)}+\zeta(\alpha)y'_n h^{1-\alpha}-\zeta(\alpha-1)y''_n h^{2-\alpha}+O\llll( h^{2} \rrrr).
\end{align}
Approximation \eqref{5_20} has second order accuracy when the function $y$ satisfies $y'(0)=y''(0)=0$. Denote
$$S_n[\alpha]=\sum_{k=1}^{n-1}\dddd{1}{k^{\alpha}}-\zeta(\alpha).$$
From \eqref{5_20} we obtain approximations \eqref{5_30} and \eqref{6_05} of the Caputo derivative
\begin{align}\label{5_30}
\dddd{1}{2\Gamma(1-\alpha)h^\alpha}\sum_{k=0}^{n}\sigma_k^{(\alpha)} y_{n-k}=y_n^{(\alpha)}+O\llll( h^{2-\alpha} \rrrr),
\end{align}
where
$\sigma_0^{(\alpha)}=1-2\zeta(\alpha),\quad \sigma_1^{(\alpha)}=\dddd{1}{2^\alpha}+2\zeta(\alpha), $
$$\sigma_k^{(\alpha)}=\dddd{1}{(k+1)^\alpha}-\dddd{1}{(k-1)^\alpha},\qquad (k=2,\cdots,n-2),$$
$$\sigma_{n-1}^{(\alpha)}=-\dddd{1}{(n-2)^\alpha}-2 S_n[\alpha]+\dddd{2n^{1-\alpha}}{1-\alpha},\; \sigma_{n}^{(\alpha)}=-\dddd{1}{(n-1)^\alpha}+2 S_n[\alpha]-\dddd{2n^{1-\alpha}}{1-\alpha}.$$
\begin{align}\label{6_05}
\dddd{1}{2\Gamma(1-\alpha)h^\alpha}\sum_{k=0}^{n}\delta_k^{(\alpha)} y_{n-k}=y_n^{(\alpha)}+O\llll( h^2 \rrrr),
\end{align}
where $\delta_0^{(\alpha)}=1-3 \zeta(\alpha)+2 \zeta(\alpha-1),\; \delta_1^{(\alpha)}=\dddd{1}{2^\alpha}+4\zeta(\alpha)-4 \zeta(\alpha-1),$
\begin{align*}
&\delta_2^{(\alpha)}=\dddd{1}{3^\alpha}-1-\zeta(\alpha)+2\zeta(\alpha-1),\;\delta_{k}^{(\alpha)}=\sigma_{k}^{(\alpha)}, \quad (k=3,\cdots,n).
\end{align*}
In section 5 we use the Fourier transform method and the Euler-Maclaurin formula
for the function $y(t)/(x-t)^{1+\alpha}$ to derive the  expansion formula
	\begin{align}\label{6_10}
\dddd{1}{h^\alpha}\sum_{k=1}^{n-1}\dddd{y(x-kh)}{k^{1+\alpha}}=&\Gamma(-\alpha)y^{(\alpha)}(x)+\dddd{ y(0)}{\alpha x^{\alpha}}-\dddd{y(0) h}{2 x^{1+\alpha}}\nonumber\\
&+\sum_{m=0}^\infty\dddd{(-1)^m}{m!}\zeta(1+\alpha-m)y^{(m)}(x)h^{m-\alpha}\\
-\sum_{m=1}^\infty \dddd{B_{2m}}{(2m)!}&\llll( \sum_{k=0}^{2m-1}\binom{2m-1}{k}\dddd{\Gamma(1+\alpha+k)}{\Gamma(1+\alpha)}\dddd{y^{(2m-1-k)}(0)}{x^{1+\alpha+k}}\rrrr)h^{2m}.\nonumber
\end{align}
From \eqref{6_10} we obtain the approximations of the Caputo derivative \eqref{6_20} and \eqref{7_10} of order $2-\alpha$ and $3-\alpha$.
\begin{equation}\label{6_20}
\dddd{1}{\Gamma(-\alpha)h^\alpha}\sum_{k=0}^{n}\sigma_k^{(\alpha)} y_{n-k}= y_n^{(\alpha)}+O\llll(h^{2-\alpha}\rrrr),
\end{equation}
where $\sigma_0^{(\alpha)}=\zeta(\alpha)-\zeta(1+\alpha),\;\sigma_1^{(\alpha)}=1-\zeta(\alpha),$
$$\sigma_k^{(\alpha)}=\dddd{1}{k^{1+\alpha}},\quad (k=2,\cdots,n-2),$$
$$\sigma_{n-1}^{(\alpha)}=\dddd{1}{(n-1)^{1+\alpha}}-n S_n [\alpha+1]+S_n[\alpha]-\dddd{n^{1-\alpha}}{\alpha(1-\alpha)},$$
$$\sigma_{n}^{(\alpha)}=(n-1)S_n[\alpha+1]-S_n[\alpha]+\dddd{n^{1-\alpha}}{\alpha(1-\alpha)}.$$
\begin{equation}\label{7_10}
\dddd{1}{\Gamma(-\alpha)h^\alpha}\sum_{k=0}^{n}\delta_k^{(\alpha)} y_{n-k}= y_n^{(\alpha)}+O\llll(h^{3-\alpha}\rrrr),
\end{equation}
where
$$\delta_0^{(\alpha)}=-\zeta(1+\alpha)+\dddd{3}{2}\zeta(\alpha)-\dddd{1}{2}\zeta(\alpha-1),\;\delta_1^{(\alpha)}=1-2\zeta(\alpha)+\zeta(\alpha-1),$$
$$\delta_2^{(\alpha)}=\dddd{1}{2^{1+\alpha}}+\dddd{1}{2}\zeta(\alpha)-\dddd{1}{2}\zeta(\alpha-1),\;\delta_{k}^{(\alpha)}=\dddd{1}{k^{1+\alpha}}\quad (k=3,\cdots,n-3),$$
\begin{align*}
\delta_{n-2}^{(\alpha)}=\dddd{1}{(n-2)^{1+\alpha}}-\dddd{1}{2} \Bigg(n(n-1)&S_n[\alpha+1]-(2 n-1)S_n[\alpha],\\
&+S_n[\alpha-1]+\frac{(\alpha+2 n-2) n^{1-\alpha}}{(\alpha-2) (\alpha-1) \alpha}\Bigg),
\end{align*}
\begin{align*}
\delta_{n-1}^{(\alpha)}=\dddd{1}{(n-1)^{1+\alpha}}+n(n-2) &S_n[\alpha+1]-2(n-1)S_n[\alpha]\\
&+S_n[\alpha-1]+\dddd{2 (\alpha+n-2) n^{1-\alpha}}{\alpha(1-\alpha)(2-\alpha)},
\end{align*}
\begin{align*}
\delta_n^{(\alpha)}=-\dddd{1}{2} \Bigg((n-1) (n-2)&S_n[\alpha+1]-(2 n-3)S_n[\alpha] \\
&+S_n[\alpha-1]+\dddd{(3 \alpha+2 n-6) n^{1-\alpha}}{\alpha(1-\alpha)(2-\alpha)}\Bigg).
\end{align*}
 The expansion formula \eqref{4_15} of the $L1$ approximation contains the term $C_1(\alpha)y''_n h^{2-\alpha}$ where $C_1(\alpha)=\zeta(\alpha-1)/\Gamma(2-\alpha)$. The expansions of approximations \eqref{5_30} and \eqref{6_20} contain terms $C_9(\alpha)y''_n h^{2-\alpha}$ and $C_{12}(\alpha)y''_n h^{2-\alpha}$ with coefficients
$$C_9(\alpha)=\frac{\zeta (\alpha)-2\zeta (\alpha-1)}{2\Gamma (1-\alpha)},\; C_{12}(\alpha)=\frac{\zeta (\alpha)-\zeta (\alpha-1)}{2\Gamma (-\alpha)}.$$
		\begin{figure}[ht]
  \centering
  \caption{Graph of the coefficients $C_1(\alpha)$(black), $C_9(\alpha)$(red), $C_{12}(\alpha)$(blue).}
  \includegraphics[width=0.5\textwidth]{graph1.eps}
\end{figure}

The accuracy of the  numerical solutions of fractional differential equations which use approximations \eqref{3_10}, \eqref{5_30} and \eqref{6_20} of the Caputo derivative is influenced by the values of the coefficients $C_1(\alpha),C_9(\alpha)$ and $C_{12}(\alpha)$. In Figure 1 we compare the coefficients $C_1(\alpha)$, $C_9(\alpha)$  and $C_{12}(\alpha)$. The three coefficients are positive and the coefficient $C_{12}(\alpha)$ of approximation \eqref{6_20} is smaller than $C_1(\alpha)$ and $C_9(\alpha)$.  In Table 1 and Table 9 we compute the maximum errors of the numerical solutions of Equation I, Equation II and Equation III which use approximations  \eqref{3_10} and \eqref{6_20}. The accuracy of the numerical solution which uses approximation \eqref{6_20} for the Caputo derivative is higher  than the  accuracy of the numerical solution using the $L1$ approximation \eqref{3_10}. The improvement is  $88\%$ for Equation I and $\alpha=0.25$, $35\%$ for Equation II and $\alpha=0.5$ and $9\%$ for Equation III with $\alpha=0.75$.
\section{Approximation for the Caputo Derivative of Order $\boldsymbol{2-\alpha}$}
In this section we use approximation \eqref{5_10} for the fractional integral  to obtain approximations for the Caputo derivative of order $1-\alpha$ and $2-\alpha$. From \eqref{5_10}
\begin{align}\label{8_10}
h^{\beta}\sum_{k=1}^{n-1} \dddd{ z_{n-k}}{k^{1-\beta}}=K^{\beta}z_n+\zeta(1-\beta) z_n h^{\alpha}-\zeta(-\beta)z'_n h^{1+\beta}
+O\llll(h^{2+\beta} \rrrr).
\end{align}
Approximation \eqref{8_10} has accuracy $O\llll(h^{2+\beta}\rrrr)$ when the function $z$ satisfies  $z(0)=z'(0)=0$. The Caputo derivative of order $\alpha$ satisfies
$$\Gamma(1-\alpha) y_n^{(\alpha)}=K^{1-\alpha}y'_n.$$
 From \eqref{8_10} with $\beta=1-\alpha$ and $z(x)=y'(x)$ we obtain
\begin{align}\label{8_20}
h^{1-\alpha}\sum_{k=1}^{n-1} \dddd{ y'_{n-k}}{k^{\alpha}}=\Gamma(1-\alpha)y^{(\alpha)}_n+\zeta(\alpha) y'_n h^{1-\alpha}-\zeta(\alpha-1)y''_nh^{2-\alpha}
+O\llll(h^{3-\alpha} \rrrr).
\end{align}
By approximating $y'_{n-k}$ in \eqref{8_20}  using central difference approximation
\begin{align}\label{9_10}
y'_{n-k}=\dddd{y_{n-k+1}-y_{n-k-1}}{2h}+\dddd{h^2}{6}y'''(\xi_{n-k}),
\end{align}
where $x_{n-k-1}<\xi_{n-k}<x_{n-k+1}$, we obtain the approximation
\begin{align}\label{9_20}
&h^{1-\alpha}\sum_{k=1}^{n-1} \dddd{ y'_{n-k}}{k^{\alpha}}\approx \dddd{1}{2h^\alpha} \sum_{k=1}^{n-1} \dddd{ y_{n-k+1}-y_{n-k-1}}{k^{\alpha}}=\dddd{1}{2h^\alpha}\llll( \sum_{k=1}^{n-1} \dddd{ y_{n-k+1}}{k^{\alpha}}-\sum_{k=1}^{n-1} \dddd{y_{n-k-1}}{k^{\alpha}}\rrrr)\nonumber\\
&=\dddd{1}{2h^\alpha}\Bigg(y_n+\dddd{y_{n-1}}{2^\alpha}+ \sum_{k=3}^{n-1} \dddd{ y_{n-k+1}}{k^{\alpha}}-\sum_{k=1}^{n-3} \dddd{y_{n-k-1}}{k^{\alpha}}-\dddd{y_{1}}{(n-2)^\alpha} -\dddd{y_{0}}{(n-1)^\alpha}\Bigg)\\
&=\dddd{1}{2h^\alpha}\llll(y_n+\dddd{y_{n-1}}{2^\alpha} +\sum_{k=2}^{n-2} y_{n-k}\llll(\dddd{1}{(k+1)^\alpha}-\dddd{1}{(k-1)^\alpha}  \rrrr)-\dddd{y_{1}}{(n-2)^\alpha} -\dddd{y_{0}}{(n-1)^\alpha} \rrrr).\nonumber
\end{align}
In Lemma 1 we show that approximation \eqref{9_20} has second order accuracy.
\begin{lem} Let $M_3=\max_{0\leq t\leq x}|y'''(t)|$. Then
$$\llll| h^{1-\alpha}\sum_{k=1}^{n-1} \dddd{ y'_{n-k}}{k^{\alpha}}- \dddd{1}{2h^\alpha} \sum_{k=1}^{n-1} \dddd{ y_{n-k+1}-y_{n-k-1}}{k^{\alpha}}\rrrr|
<\llll(\dddd{M_3 x^{1-\alpha}}{6(1-\alpha)}\rrrr)h^{2}.$$
\end{lem}
\begin{proof} From \eqref{9_10}
\begin{align*}
\Bigg| h^{1-\alpha}\sum_{k=1}^{n-1}\dddd{ y'_{n-k}}{k^{\alpha}}-\dddd{1}{2h^\alpha} \sum_{k=1}^{n-1} &\dddd{ y_{n-k+1}-y_{n-k-1}}{k^{\alpha}}\Bigg|=\dddd{h^{3-\alpha}}{6}\llll|\sum_{k=1}^{n-1} \dddd{y'''(\xi_{n-k})}{k^\alpha}  \rrrr|\\
&<\dddd{h^{3-\alpha}M_3}{6}\sum_{k=1}^{n-1} \dddd{1}{k^\alpha}<\dddd{h^{3-\alpha}M_3}{6}\int_0^n \dddd{1}{t^\alpha}d t\\
&<\dddd{h^{3-\alpha}M_3 n^{1-\alpha}}{6(1-\alpha)}=\dddd{M_3 x^{1-\alpha}h^{2}}{6(1-\alpha)}.
\end{align*}
\end{proof}
Let
\begin{align}\label{9_30}
\mathcal{A}_n^\omega[y]=\dddd{1}{2\Gamma(1-\alpha)h^\alpha}\sum_{k=0}^{n}\omega_k^{(\alpha)} y_{n-k},
\end{align}
where
$\omega_0^{(\alpha)}=1,\quad \omega_1^{(\alpha)}=\dddd{1}{2^\alpha},\quad \omega_{n-1}^{(\alpha)}=-\dddd{1}{(n-2)^\alpha},\quad \omega_{n}^{(\alpha)}=-\dddd{1}{(n-1)^\alpha}$,
$$\omega_k^{(\alpha)}=\dddd{1}{(k+1)^\alpha}-\dddd{1}{(k-1)^\alpha},\quad (k=2,\cdots,n-2).$$
From \eqref{8_20} and \eqref{9_20} we obtain the expansion formula of approximation $\mathcal{A}_n^\omega[y]$.
	\begin{lem} Let $y(0)=y'(0)=0$. Then
\begin{align}\label{10_10}
\mathcal{A}_n^\omega[y]=y_n^{(\alpha)}+\dddd{\zeta(\alpha)}{\Gamma(1-\alpha)}y'_n h^{1-\alpha}-\dddd{\zeta(\alpha-1)}{\Gamma(1-\alpha)}y''_n h^{2-\alpha}+O\llll( h^{2} \rrrr).
\end{align}
	\end{lem}
	The numbers $\omega_k^{(\alpha)}$ satisfy $\sum_{k=0}^{n}\omega_k^{(\alpha)}=0$ and  $\mathcal{A}_n^\omega[1]=D^\alpha 1=0$. From Lemma 2, approximation $\mathcal{A}_n^\omega[y]$ for the Caputo derivative has order $1-\alpha$.
\begin{align}\label{10_20}
\dddd{1}{2\Gamma(1-\alpha)h^\alpha}\sum_{k=0}^{n}\omega_k^{(\alpha)} y_{n-k}=y_n^{(\alpha)}+O\llll( h^{1-\alpha} \rrrr).
\end{align}
 When $n=2$, approximation $\mathcal{A}_2^\omega[y]$ has weights $\omega_0^{(\alpha)}=1,\omega_1^{(\alpha)}=0,\omega_2^{(\alpha)}=-1$. Denote by Equation I, Equation II and Equation III the fractional relaxation equations
\begin{align}\tag{{\bf Equation I}}
y^{(\alpha)}(x)+y(x)=1&+x+x^2+x^3+x^4+\dddd{x^{1-\alpha}}{\Gamma(2-\alpha)}\nonumber\\
&+\dddd{2 x^{2-\alpha}}{\Gamma(3-\alpha)}
+\dddd{6 x^{3-\alpha}}{\Gamma(4-\alpha)}+\dddd{24 x^{4-\alpha}}{\Gamma(5-\alpha)},\; y(0)=1.\nonumber
\end{align}
Equation I has the solution $y(x)=1+x+x^2+x^3+x^4$.
\begin{align} \tag{{\bf Equation II}}
y^{(\alpha)}(x)+y(x)=e^x+x^{1-\alpha}E_{1,2-\alpha}(x),\quad y(0)=1.
\end{align}
Equation II has the solution $y(x)=e^x$.
\begin{align}
&y^{(\alpha)}(x)+y(x)=\cos (2\pi x)+i \pi x^{1-\alpha}
\llll(E_{1,2-\alpha}(2\pi i x)-E_{1,2-\alpha}(-2\pi i x)\rrrr),\nonumber\\
& y(0)=1. \tag{{\bf Equation III}}\nonumber
\end{align}
Equation III has the solution $y(x)=\cos (2\pi x)$. The numerical results for the maximum error and order of numerical solution $NS[20]$ of Equation I, Equation II and Equation III are given in Table 2.
\setlength{\tabcolsep}{0.5em}
{ \renewcommand{\arraystretch}{1.1}
\begin{table}[ht]
	\caption{Error and order of numerical solution $NS[20]$ of Equation I with $\alpha=0.25$, Equation II with $\alpha=0.5$ and Equation III with $\alpha=0.75$.}
	\small
	\centering
  \begin{tabular}{ l | c  c | c  c | c  c }
		\hline
		\hline
		\multirow{2}*{ $\quad \boldsymbol{h}$}  & \multicolumn{2}{c|}{{\bf Equation I}} & \multicolumn{2}{c|}{{\bf Equation II}}  & \multicolumn{2}{c}{{\bf Equation III}} \\
		\cline{2-7}
   & $Error$ & $Order$  & $Error$ & $Order$  & $Error$ & $Order$ \\
		\hline \hline
$0.003125$    & $0.040435$  & $0.7606$  & $0.061232$   & $0.5206$    & $0.521231$   & $0.3076$       \\
$0.0015625$   & $0.023913$  & $0.7578$  & $0.042851$   & $0.5150$    & $0.423886$   & $0.2983$       \\
$0.00078125$  & $0.014162$  & $0.7558$  & $0.030075$   & $0.5108$    & $0.346614$   & $0.2904$        \\
$0.000390625$ & $0.008395$  & $0.7544$  & $0.021152$   & $0.5077$    & $0.284739$   & $0.2837$        \\
\hline
  \end{tabular}
	\end{table}
	}

	By substituting $y'_n$ in \eqref{10_10} using the approximation
$$y'_{n}=\dddd{y_{n}-y_{n-1}}{h}+\dddd{h}{2}y''_n+O\llll( h^2 \rrrr),$$
we obtain the second-order expansion formula
\begin{align}\label{11_10}
\mathcal{A}_n^{\bar{\sigma}}[y]=\dddd{1}{2\Gamma(1-\alpha)h^\alpha}\sum_{k=0}^{n}\bar{\sigma}_k^{(\alpha)} y_{n-k}= y_n^{(\alpha)}+\Bigg(&\dddd{\zeta(\alpha)-2\zeta(\alpha-1)}{2\Gamma(1-\alpha)}\Bigg)y''_n h^{2-\alpha}\nonumber\\
&+O\llll( h^{2} \rrrr),
\end{align}
where $\bar{\sigma}_0^{(\alpha)}=1-2\zeta(\alpha),\quad \bar{\sigma}_1^{(\alpha)}=\dddd{1}{2^\alpha}+2\zeta(\alpha)$, $\bar{\sigma}_k^{(\alpha)}=\omega_k^{(\alpha)},\; (k=2,\cdots,n)$.

The sum of the weights $\bar{\sigma}_{k}^{(\alpha)}$ of approximation $\mathcal{A}_n^{\bar{\sigma}}[y]$  is zero and
$$\mathcal{A}_n^{\bar{\sigma}}[1]=D^\alpha 1 =0.$$
  Approximation $\mathcal{A}_n^{\bar{\sigma}}[y]$ has accuracy  $O\llll( h^{2-\alpha} \rrrr)$ when  $y^\prime (0)=0$. Denote
$$W_n[\alpha]=\sum _{k=1}^{n-1} \frac{1}{k^\alpha}-\frac{n^{1-\alpha}}{1-\alpha}-\zeta (\alpha).$$
\begin{clm} Let $y(x)=x$. Then
$$\mathcal{A}_n^{\bar{\sigma}}[y(x)]-y^{(\alpha)}(x)=\dddd{W_n[\alpha]h^{1-\alpha}}{\Gamma(1-\alpha)}.$$
\end{clm}
\begin{proof}
$$2\Gamma(1-\alpha)h^\alpha \mathcal{A}_n^{\bar{\sigma}}[x]=\sum_{k=0}^{n}\bar{\sigma}_k^{(\alpha)}(x-k h) =-h\sum_{k=1}^{n}k\bar{\sigma}_k^{(\alpha)},$$
\begin{align*}
\mathcal{A}_n^{\bar{\sigma}}[x]=-\dddd{h^{1-\alpha}}{2\Gamma(1-\alpha)}\Bigg( \dddd{1}{2^\alpha}+2\zeta(\alpha)+\sum_{k=2}^{n-2}\Bigg( &\dddd{k}{(k+1)^\alpha}-  \dddd{k}{(k-1)^\alpha} \Bigg) \\
&-\dddd{n-1}{(n-2)^\alpha} -\dddd{n}{(n-1)^\alpha}\Bigg).
\end{align*}
By changing the index of summation we obtain
$$\mathcal{A}_n^{\bar{\sigma}}[x]=\dddd{h^{1-\alpha}}{\Gamma(1-\alpha)} \left(\sum _{k=1}^{n-1} \dddd{1}{k^\alpha}-\zeta (\alpha)\right).$$
Then
\begin{align*}
\mathcal{A}_n^{\bar{\sigma}}[y(x)]-y^{(\alpha)}(x)&=\dddd{h^{1-\alpha}}{\Gamma(1-\alpha)} \left(\sum _{k=1}^{n-1} \dddd{1}{k^\alpha}-\zeta (\alpha)\right)-\dddd{x^{1-\alpha}}{\Gamma(2-\alpha)}\\
&=\dddd{h^{1-\alpha}}{\Gamma(1-\alpha)} \left(\sum _{k=1}^{n-1} \frac{1}{k^\alpha}-\zeta (\alpha)-\frac{n^{1-\alpha}}{1-\alpha}\right)=\dddd{h^{1-\alpha}W_n[\alpha]}{\Gamma(1-\alpha)}.
\end{align*}
\end{proof}
Now we derive an approximation for the Caputo derivative of order $2-\alpha$ by modifying the weights of approximation $\mathcal{A}_n^{\bar{\sigma}}[y]$ with index $n-1$ and $n$.
$$y(x)=y(x)-y(0)-y'(0)x+y(0)+y'(0)x=z(x)+y(0)+y'(0)x.$$
The function $z(x)=y(x)-y(0)-y'(0)x$ satisfies $z(0)=z'(0)=0$. Then
$$\mathcal{A}^{\bar{\sigma}}_n[z]=z^{(\alpha)}(x)+O\llll( h^{2-\alpha} \rrrr)=y^{(\alpha)}(x)-y'(0)\dddd{x^{1-\alpha}}{\Gamma(2-\alpha)}+O\llll( h^{2-\alpha} \rrrr).$$
We use Claim 3 to determine the first term of the expansion of approximation $\mathcal{A}_n^{\bar{\sigma}}[y]$ at the initial point $x=0$.
$$\mathcal{A}^{\bar{\sigma}}_n[y]=\mathcal{A}^{\bar{\sigma}}_n[z]+\mathcal{A}^{\bar{\sigma}}_n[y(0)+y'(0)x],$$
$$\mathcal{A}^{\bar{\sigma}}_n[y]=y^{(\alpha)}(x)-y'(0)\dddd{x^{1-\alpha}}{\Gamma(2-\alpha)}+y'(0)\mathcal{A}_n[x]+O\llll( h^{2-\alpha} \rrrr),$$
\begin{align*}
\dddd{1}{2\Gamma(1-\alpha)h^\alpha}\sum_{k=0}^{n}\bar{\sigma}_k y_{n-k}=y_n^{(\alpha)}+\llll(\dddd{y'_0 W_n[\alpha]}{\Gamma(1-\alpha)}\rrrr)h^{1-\alpha}+O\llll( h^{2-\alpha} \rrrr).
\end{align*}
By approximating $y'_0$ using  forward difference
$y'_0=(y_1-y_{0})/h+O(h)$ we obtain  approximation \eqref{5_30} for the Caputo derivative
\begin{align*}
\mathcal{A}_n^{\sigma}[y]=\dddd{1}{2\Gamma(1-\alpha)h^\alpha}\sum_{k=0}^{n}\sigma_k^{(\alpha)} y_{n-k}= y_n^{(\alpha)}+O\llll( h^{2-\alpha} \rrrr),
\end{align*}
where $\sigma_k^{(\alpha)}=\bar{\sigma}_k^{(\alpha)}$ for $k=0,\cdots,n-2$ and
$$\sigma_{n-1}^{(\alpha)}=-\dddd{1}{(n-2)^\alpha}-2 W_n[\alpha],\; \sigma_{n}^{(\alpha)}=-\dddd{1}{(n-1)^\alpha}+2 W_n[\alpha].$$
When $n=2$, approximation $\mathcal{A}_n^{\sigma}[y]$ has weights
$$\sigma_{0}^{(\alpha)}=1-2 \zeta (\alpha),\; \sigma_{1}^{(\alpha)}=4 \zeta (\alpha)+\frac{2^{2-\alpha}}{1-\alpha}-2,\; \sigma_{2}^{(\alpha)}=-2 \zeta (\alpha)-\frac{2^{2-\alpha}}{1-\alpha}+1.$$
\setlength{\tabcolsep}{0.5em}
{ \renewcommand{\arraystretch}{1.1}
\begin{table}[ht]
	\caption{Error and order of numerical solution $NS[9]$ of Equation I with $\alpha=0.25$, Equation II with $\alpha=0.5$ and Equation III with $\alpha=0.75$.}
	\small
	\centering
  \begin{tabular}{ l | c  c | c  c | c  c }
		\hline
		\hline
		\multirow{2}*{ $\quad \boldsymbol{h}$}  & \multicolumn{2}{c|}{{\bf Equation I}} & \multicolumn{2}{c|}{{\bf Equation II}}  & \multicolumn{2}{c}{{\bf Equation III}} \\
		\cline{2-7}
   & $Error$ & $Order$  & $Error$ & $Order$  & $Error$ & $Order$ \\
		\hline \hline
$0.003125$    & $0.0000673$          & $1.6772$  & $0.0000637$          & $1.4780$    & $0.0026324$   & $1.2410$       \\
$0.0015625$   & $0.0000208$          & $1.6914$  & $0.0000227$          & $1.4846$    & $0.0011108$   & $1.2448$       \\
$0.00078125$  & $6.4\times 10^{-6}$  & $1.7025$  & $8.1\times 10^{-6}$  & $1.4892$    & $0.0004680$   & $1.2470$        \\
$0.000390625$ & $2.0\times 10^{-6}$  & $1.7112$  & $2.9\times 10^{-6}$  & $1.4924$    & $0.0001970$   & $1.2483$        \\
\hline
  \end{tabular}
	\end{table}
	}

The accuracy of the $L1$ approximation \eqref{3_10} and approximation \eqref{5_30} is $O\llll(h^{2-\alpha}\rrrr)$. Now we show that the weights of approximation \eqref{5_30} satisfy \eqref{3_20}.	The Riemann zeta function is negative and decreasing on the interval $[0,1]$. The weight $\sigma_0^{(\alpha)}=1-\zeta(\alpha)$ of approximation \eqref{5_30} is positive and  $\sigma_1^{(\alpha)}=\dddd{1}{2^\alpha}+\zeta(\alpha)$ is decreasing as a function of $\alpha$. Then $\sigma_1^{(\alpha)}<\sigma_1^{(0)}=0$.
	From the binomial formula,
	$$\sigma_k^{(\alpha)}=\llll(\dddd{1}{k^\alpha}-\dddd{\alpha}{k^{1+\alpha}}\rrrr)-\llll(\dddd{1}{k^\alpha}+\dddd{\alpha}{k^{1+\alpha}}\rrrr)+O\llll(\dddd{1}{k^{3+\alpha}}\rrrr)=\dddd{2\alpha}{k^{1+\alpha}}+O\llll(\dddd{1}{k^{3+\alpha}}\rrrr)$$
The number $\sigma_k^{(\alpha)}\approx \dddd{2\alpha}{k^{1+\alpha}}$ when  $k$ is a sufficiently large positive integer. In Claim 4 and Claim 5 we obtain estimates for the weights $\sigma_{n-1}^{(\alpha)}$ and $\sigma_n^{(\alpha)}$
The Bernoulli inequality holds when $r<0$ and $x>-1$.
$$(1+x)^r>1+r x.$$
					\begin{clm}
$$\sigma_{n-1}^{(\alpha)}<-\dddd{11\alpha}{6n^{1+\alpha}}.$$
			\end{clm}
			\begin{proof}  From the Bernoulli inequality
			$$\dddd{1}{(n-2)^\alpha}=n^{-\alpha}\llll(1-\dddd{2}{n}\rrrr)^{-\alpha}>n^{-\alpha}\llll(1+\dddd{2\alpha}{n}\rrrr)=\dddd{1}{n^\alpha}+\dddd{2\alpha}{n^{1+\alpha}}.$$
			From the formula for the sum of powers
\begin{align}\label{14_05}
\sum _{k=1}^{n-1} \frac{1}{k^\alpha}=\zeta (\alpha)+\frac{n^{1-\alpha}}{1-\alpha}\sum_{m=0}^\infty \binom{1-\alpha}{m}\dddd{B_m}{n^m},
\end{align}
we obtain the following estimates for $S_n(\alpha)$
	\begin{align}\label{14_10}
	\dddd{n^{1-\alpha}}{1-\alpha}-\dddd{1}{2 n^\alpha}-\dddd{\alpha}{12 n^{1+\alpha}}<\sum _{k=1}^{n-1} \frac{1}{k^\alpha}-\zeta(\alpha)<\dddd{n^{1-\alpha}}{1-\alpha}-\dddd{1}{2 n^\alpha}.
	\end{align}
	Then
	$$\sigma_{n-1}^{(\alpha)}=-\dddd{1}{(n-2)^\alpha}-2 \left(S_n[\alpha]-\frac{n^{1-\alpha}}{1-\alpha}\right)<-\dddd{1}{(n-2)^\alpha}+\dddd{1}{n^\alpha}+\dddd{\alpha}{6n^{1+\alpha}}.$$
	From \eqref{14_10}
		$$\sigma_{n-1}^{(\alpha)}<-\dddd{1}{n^\alpha}-\dddd{2\alpha}{n^{1+\alpha}}+\dddd{1}{n^\alpha}+\dddd{\alpha}{6n^{1+\alpha}}=-\dddd{11\alpha}{6n^{1+\alpha}}.$$
			\end{proof}
				\begin{clm}
$$-\dddd{2}{(n-1)^\alpha}<\sigma_{n}^{(\alpha)}<-\dddd{2}{n^\alpha}.$$
			\end{clm}
			\begin{proof} From \eqref{14_10}
			$$\sigma_{n}^{(\alpha)}=-\dddd{1}{(n-1)^\alpha}+2 \left(S_n[\alpha]-\frac{n^{1-\alpha}}{1-\alpha}\right)<-\dddd{1}{(n-1)^\alpha}-\dddd{1}{n^\alpha}<-\dddd{2}{n^\alpha}.$$
From the Bernoulli inequality
\begin{align}\label{15_10}
	\dddd{1}{(n-1)^\alpha}=n^{-\alpha}\llll(1-\dddd{1}{n}\rrrr)^{-\alpha}>n^{-\alpha}\llll(1+\dddd{\alpha}{n}\rrrr)=\dddd{1}{n^\alpha}+\dddd{\alpha}{n^{1+\alpha}}.
	\end{align}
From \eqref{14_10} and \eqref{15_10}
				$$\sigma_{n}^{(\alpha)}>-\dddd{1}{(n-1)^\alpha}-\dddd{1}{n^\alpha}-\dddd{\alpha}{6 n^{1+\alpha}}>-\dddd{2}{(n-1)^\alpha}.$$
			\end{proof}
	\section{ Second-order approximation for the Caputo derivative}
	We use approximation \eqref{11_10} to derive a second-order approximation for the Caputo derivative. By approximating $y''_n$ in \eqref{11_10} using a second-order backward difference approximation
$y''_n=(y_n-2y_{n-1}+y_{n-2})/h^2+O\llll( h \rrrr),$
we obtain the  approximation for the Caputo derivative
\begin{align}
\dddd{1}{2\Gamma(1-\alpha)h^\alpha}\sum_{k=0}^{n}\bar{\delta}_k^{(\alpha)} y_{n-k}= y_n^{(\alpha)}+O\llll(h^2\rrrr),
\end{align}\label{15_20}
where $\bar{\delta}_k^{(\alpha)}=\bar{\sigma}_k^{(\alpha)}$ for $k=3,\cdots,n$ and
$$\bar{\delta}_0^{(\alpha)}=\bar{\sigma}_0^{(\alpha)}-\zeta(\alpha)+2\zeta(\alpha-1)=1-3 \zeta(\alpha)+2 \zeta(\alpha-1),$$
$$\bar{\delta}_1^{(\alpha)}=\bar{\sigma}_1^{(\alpha)}+2\zeta(\alpha)-4\zeta(\alpha-1)=\dddd{1}{2^\alpha}+4\zeta(\alpha)-4 \zeta(\alpha-1),$$
$$\bar{\delta}_2^{(\alpha)}=\bar{\sigma}_2^{(\alpha)}-\zeta(\alpha)+2\zeta(\alpha-1)=\dddd{1}{3^\alpha}-1-\zeta(\alpha)+2\zeta(\alpha-1).$$
Approximation \eqref{19_20} has second-order accuracy  when $y'(0)=0$.
By using the method from the previous section we obtain the second-order approximation
\begin{align*}
\mathcal{A}_n^\delta[y]=\dddd{1}{2\Gamma(1-\alpha)h^\alpha}\sum_{k=0}^{n}\delta_k^{(\alpha)} y_{n-k}=y_n^{(\alpha)}+O\llll( h^2 \rrrr),
\end{align*}
where $\delta_k^{(\alpha)}=\bar{\delta}_k^{(\alpha)}$ for $k=0,\cdots,n-2$ and
\begin{align*}
\delta_{n-1}^{(\alpha)}=-\dddd{1}{(n-2)^\alpha}-2 W_n[\alpha],\;\delta_{n}^{(\alpha)}=-\dddd{1}{(n-1)^\alpha}+2 W_n[\alpha].
\end{align*}
Approximation $\mathcal{A}_n^\delta[y]$ has a second-order accuracy $O\llll(h^2\rrrr)$ and accuracy $O\llll( h^{2-\alpha} \rrrr)$ when $n$ is a {\it small} positive number. When $n=2$, approximation $\mathcal{A}_n^\delta[y]$ has weights
$\delta_{0}^{(\alpha)}=1+2 \zeta (\alpha-1)-3 \zeta (\alpha)$ and
$$\delta_{1}^{(\alpha)}=-4 \zeta (\alpha-1)+6 \zeta (\alpha)+\frac{2^{2-\alpha}}{1-\alpha}-2,\; \delta_{2}^{(\alpha)}=1+2 \zeta (\alpha-1)-3 \zeta (\alpha)-\frac{2^{2-\alpha}}{1-\alpha}.$$
When $n=3$: $\delta_{0}^{(\alpha)}=1+2 \zeta (\alpha-1)-3 \zeta (\alpha),\;\delta_{1}^{(\alpha)}=\dddd{1}{2^\alpha}-4 \zeta (\alpha-1)+4 \zeta (\alpha),$
$$\delta_{2}^{(\alpha)}=2 \zeta (\alpha-1)+\zeta (\alpha)-2\llll(\dddd{1}{2^\alpha}-\dddd{ 3^{1-\alpha}}{1-\alpha}\rrrr)-3,\;\delta_{3}^{(\alpha)}=2-2 \zeta (\alpha)-\dddd{2\times 3^{1-\alpha}}{1-\alpha}+\dddd{1}{2^\alpha}.$$
In Table 4 we present the numerical results for the numerical solutions of  Equation I, Equation II and Equation III using approximation $\mathcal{A}_n^\delta[y]$  for the Caputo derivative. In Figure 2 we compare the numerical solutions of Equation III using approximations \eqref{10_20}, \eqref{5_30} and $\mathcal{A}_n^\delta[y]$  when $\alpha=0.6$.
\setlength{\tabcolsep}{0.5em}
{ \renewcommand{\arraystretch}{1.1}
\begin{table}[ht]
	\caption{Error and order of numerical solution $NS[10]$ of Equation I and $\alpha=0.25$, Equation II and $\alpha=0.5$ and Equation III,$\alpha=0.75$.}
	\small
	\centering
  \begin{tabular}{ l | c  c | c  c | c  c }
		\hline
		\hline
		\multirow{2}*{ $\quad \boldsymbol{h}$}  & \multicolumn{2}{c|}{{\bf Equation I}} & \multicolumn{2}{c|}{{\bf Equation II}}  & \multicolumn{2}{c}{{\bf Equation III}} \\
		\cline{2-7}
   & $Error$ & $Order$  & $Error$ & $Order$  & $Error$ & $Order$ \\
		\hline \hline
$0.003125$    & $0.0000171$         & $1.9751$  & $2.1\times 10^{-6}$   & $1.9293$    & $0.0001178$          & $1.9776$       \\
$0.0015625$   & $4.3\times 10^{-6}$  & $1.9854$  & $5.4\times 10^{-7}$   & $1.9518$    & $0.0000297$          & $1.9871$       \\
$0.00078125$  & $1.1\times 10^{-6}$  & $1.9914$  & $1.4\times 10^{-7}$   & $1.9668$    & $7.5\times 10^{-6}$  & $1.9924$        \\
$0.000390625$ & $2.8\times 10^{-7}$  & $1.9949$  & $3.5\times 10^{-8}$   & $1.9770$    & $1.9\times 10^{-6}$  & $1.9955$        \\
\hline
  \end{tabular}
	\end{table}
	}
			\begin{figure}[ht]
  \centering
  \caption{Graph of the exact solution of Equation III  and  numerical solutions $NS[9]$-red,  and $NS[10]$-blue, and $NS[20]$-green for $h=0.1$ and $\alpha=0.6$.}
  \includegraphics[width=0.55\textwidth]{graph2.eps}
\end{figure}

Approximations \eqref{4_20} and \eqref{6_05}  have  second-order accuracy and their weights satisfy \eqref{4_30}.  In Lemma 7 and Lemma 8 we prove that $\delta_0^{(\alpha)}>0$ and $\delta_1^{(\alpha)}<0$. In the proof of Lemma 7 and Lemma 8 we use the estimates from Claim 6 and the  Laurent expansion of the Riemann zeta function
$$\zeta(\alpha)=\dddd{1}{\alpha-1}+\sum_{n=0}^{\infty}(-1)^n \dddd{\gamma_n}{n!}(\alpha-1)^n,$$
where $\gamma_n$ are the the Stieltjes constants
$$\gamma_n=\lim_{m\rightarrow \infty}\llll(\sum_{k=1}^{m}\dddd{\ln^n k}{k}-\dddd{\ln^{n+1}m}{n+1}\rrrr).$$
The Stieltjes constant $\gamma_0= 0.57721$ is equal to the Euler-Mascheroni constant and $\gamma_1= -0.0728<0,\; \gamma_2= -0.00969<0,\; \gamma_3= 0.002>0 $.
\begin{clm} Let $0<\alpha<1$. Then
$$\dddd{1}{1-\alpha}>1+\alpha+\alpha^2,\quad \dddd{2}{2-\alpha}<1+\alpha,$$
$$\dddd{1}{2^\alpha}<1-\alpha\ln 2+\dddd{\ln 2}{2}\alpha^2
,\quad \dddd{1}{3^\alpha}>1-\alpha\ln 3.$$
\end{clm}
The estimates of Claim 6 are obtained from the Maclaurin series  expansion of the functions $1/(1-\alpha),2^{-\alpha}=e^{-\alpha \ln 2}$ and $3^{-\alpha}=e^{-\alpha \ln 3}$.
\begin{lem}
$$\delta_1^{(\alpha)}=\dddd{1}{2^\alpha}+4\zeta(\alpha)-4 \zeta(\alpha-1)<0.$$
\end{lem}
\begin{proof} From the Laurent expansion of the Riemann zeta function
\begin{align}\label{18_10}
\zeta(\alpha)<\dddd{1}{\alpha-1}+\gamma_0-\gamma_1 (\alpha-1),
\end{align}
\begin{align}\label{18_20}
\zeta(\alpha-1)>\dddd{1}{\alpha-2}+\gamma_0-\gamma_1 (\alpha-2)+\dddd{\gamma_2}{2} (\alpha-2)^2.
\end{align}
Then
$$\delta_1^{(\alpha)}<\dddd{1}{2^\alpha}+\dddd{4}{\alpha-1}-\dddd{4}{\alpha-2}-4\gamma_1 -2\gamma_2 (\alpha-2)^2<0.$$
We have that
$$-4\gamma_1 -2\gamma_2 (\alpha-2)^2<-4\gamma_1-8\gamma_2<0.37.$$
Then
$$\delta_1^{(\alpha)}<\dddd{1}{2^\alpha}-\dddd{4}{1-\alpha}+\dddd{4}{2-\alpha}+0.37.$$
From Claim 6
$$\delta_1^{(\alpha)}<1-\alpha\ln 2+\dddd{\ln 2}{2}\alpha^2-4(1+\alpha+\alpha^2)+2(1+\alpha)+0.37,$$
$$\delta_1^{(\alpha)}<-\dddd{1}{2}\llll(8-\ln 2\rrrr)\alpha^2-(2+\ln 2)\alpha-0.63. $$
The function $-\dddd{1}{2}\llll(8-\ln 2\rrrr)\alpha^2-(2+\ln 2)\alpha$ is decreasing on $[0,1]$ . Then
$$\delta_1^{(\alpha)}<\delta_1^{(0)}<-0.63<0.$$
\end{proof}
\begin{lem}
$$\delta_2^{(\alpha)}=\dddd{1}{3^\alpha}-1-\zeta(\alpha)+2\zeta(\alpha-1)>0.$$
\end{lem}
\begin{proof} From \eqref{18_10} and \eqref{18_20}
$$\delta_2^{(\alpha)}>\dddd{1}{3^\alpha}-1+\dddd{1}{1-\alpha}-\dddd{2}{2-\alpha}+\gamma_0+\gamma_1 (3-\alpha)+\gamma_2 (\alpha-2)^2.$$
The numbers $\gamma_1$ and $\gamma_2$ are negative. Then
$$\gamma_0+\gamma_1 (3-\alpha)+\gamma_2 (\alpha-2)^2>\gamma_0+3\gamma_1+4\gamma_2>0.32,$$
$$\delta_2^{(\alpha)}>\dddd{1}{3^\alpha}-1+\dddd{1}{1-\alpha}-\dddd{2}{2-\alpha}+0.32.$$
From Claim 6
$$\delta_2^{(\alpha)}>1-\alpha \ln 3-1+1+\alpha+\alpha^2-(1+\alpha)+0.32,$$
$$\delta_2^{(\alpha)}>\alpha^2-\alpha \ln 3+0.32.$$
The function $\alpha^2-\alpha \ln 3$ has a minimum at the point $\alpha=\ln 3/2$. Then
$$\delta_2^{(\alpha)}>0.32-\dddd{\ln^2 3}{4}>0.018>0.$$
\end{proof}
	\section{Numerical Solution of the Fractional Relaxation Equation}
The fractional relaxation equation  is a linear ordinary fractional differential equation of order $\alpha$, where $0<\alpha<1$.
	\begin{align}\label{19_10}
	y^{(\alpha)}(x)+D y(x)=F(x),\quad y(0)=y_0.
	\end{align}
	The analytical and numerical solutions of the fractional relaxation equation are discussed in \cite{Diethelm2010,Dimitrov2014,Dimitrov2016,GulsuOzturkAnapali2013,Podlubny1999,Mainardi1996}.
	In this section we derive the numerical solution of the fractional relaxation equation \eqref{19_10}. In Theorem 9 and Theorem 10 we prove that the numerical solution of equation \eqref{19_10} on the interval $[0,X]$ which uses  approximations \eqref{3_10}, \eqref{5_30} and \eqref{6_20} for the Caputo derivative converges to the exact solution when $D>D_0$, where $D_0$ is a negative number. In \cite{Dimitrov2015} we showed that
\begin{align}\label{19_20}
\bar{y}_1=\dddd{y(0)+\Gamma(2-\alpha)h^\alpha F(h)}{1+\Gamma(2-\alpha)D h^\alpha}
\end{align}
is an approximation for the value of the exact solution $y(h)$ of equation \eqref{19_10} with accuracy $O\llll(h^2\rrrr)$. Let $h=X/n$ and
\begin{align}\label{20_10}\tag{*}
	\dddd{1}{h^\alpha}\llll(\lambda_0y_m-\sum_{k=1}^{m}\lambda_k y_{m-k} \rrrr)\approx y_m^{(\alpha)},
	\end{align}
be an approximation for the Caputo derivative.
By approximating the Caputo derivative in equation \eqref{19_10} at the point $x_m=m h$ we obtain
	$$\dddd{1}{h^\alpha}\llll(\lambda_0 y_m-\sum_{k=1}^{m}\lambda_k y_{m-k} \rrrr)+Dy_m\approx F_m,$$
$$\llll(\lambda_0+Dh^\alpha\rrrr) y_m-\sum_{k=1}^{m}\lambda_k y_{m-k}\approx h^\alpha F_m.$$
	The numerical solution $\{u_m\}_{m=0}^n$ of equation \eqref{19_10}, where $u_m\approx y_m$ is computed with $u_0=y_0, u_1=\bar{y}_1$ and
\begin{align}\tag{$\boldsymbol{NS[*]}$}
		u_m=\dddd{1}{\lambda_0+Dh^\alpha}\llll(h^\alpha F_m+\sum_{k=1}^{m}\lambda_k u_{m-k}\rrrr).
		\end{align}
Suppose that approximation \eqref{20_10} has accuracy $O\llll(h^{2-\alpha}\rrrr)$, where the weights  $\lambda_k$ are positive for all $k=0,1,\cdots,m$ and satisfy
\begin{align}\label{20_20}
	\sum_{k=1}^{m}\lambda_k=\lambda_0,\quad 0<\dddd{L}{m^\alpha}<\lambda_m.
	\end{align}
The weights of approximations \eqref{3_10}, \eqref{5_30} and \eqref{6_20} satisfy \eqref{20_20}.	Let  $E_m h^{2-\alpha}$ be the error of approximation \eqref{20_10} at the point $x_m=mh$, and $E$ be a positive number such that $|E_m|<E$ for all $m=2,3,\cdots,n$. The error  $e_m=u_m-y_m$ of numerical solution $NS(*)$ satisfies $e_0=0, e_1=\bar{y}_1-y(h)$ and
			$$e_m=\dddd{1}{\lambda_0+D h^\alpha}\sum_{k=1}^{m}\lambda_k e_{m-k}+A_m h^2,$$
where $A_m=\dddd{E_m}{\lambda_0+Dh^\alpha}$. Let $A>2E/\lambda_0$ be a positive number such that $|e_1|<A h^{2-\alpha}$. When $D>0$ the numbers $A_m$ satisfy
			$$|A_m|<\dddd{|E_m|}{\lambda_0}<\dddd{E}{\lambda_0}<A.$$
When $D<0$ 	and $|D| h^\alpha<\lambda_0/2$
	$$|A_m|=\dddd{|E_m|}{\lambda_0+Dh^\alpha}<\dddd{2|E_m| }{\lambda_0}<\dddd{2E}{\lambda_0}<A.$$
			\begin{thm} 	Let $D>0$ and $C>Max\{A,A(\lambda_0+D)/D\}$. The error of numerical solution $NS[*]$ of equation \eqref{19_10} on the interval $[0,X]$ satisfies
			$$|e_m|<Ch^{2-\alpha}.$$
			\end{thm}
			\begin{proof} Induction on $m$.
			Suppose that $|e_k|<Ch^{2-\alpha}$ for $k=1,\cdots,m-1$. The error $e_m$ satisfies
$$|e_m|\leq\dddd{1}{\lambda_0+Dh^\alpha}\sum_{k=1}^{m-1}\lambda_k |e_{m-k}|+|A_m| h^2.$$
From the induction assumption
$$|e_m|<\dddd{Ch^{2-\alpha}}{\lambda_0+D h^\alpha}\sum_{k=1}^{m-1}\lambda_k +A h^2,$$
$$|e_m|<\dddd{C \lambda_0h^{2-\alpha}}{\lambda_0+Dh^\alpha} +A h^2=\llll( \dddd{C \lambda_0+A\llll(\lambda_0+D h^\alpha\rrrr)h^{\alpha}}{\lambda_0+Dh^\alpha} \rrrr)h^{2-\alpha},$$
$$|e_m|<\llll( \dddd{\lambda_0+A\llll(\lambda_0+D\rrrr)h^{\alpha}/C}{\lambda_0+Dh^\alpha} \rrrr)C h^{2-\alpha}.$$
We have that $A(\lambda_0+D)/C<D$, because $C>A(\lambda_0+D)/D$. Hence
$$|e_m|<C h^{2-\alpha}.$$
	\end{proof}
			\begin{thm} Let $- L/X^\alpha<D<0$. The error of numerical solution $NS[*]$ of equation \eqref{19_10} on the interval $[0,X]$ satisfies
			$$|e_m|<Ch^{2-\alpha}$$
			where $C=Max\llll\{A,\dddd{A X^\alpha (\lambda_0+D)}{L-D X^\alpha}\rrrr\}$.
			\end{thm}
			\begin{proof} Induction on $m$.
			Suppose that $|e_k|<Ch^{2-\alpha}$ for $k=1,\cdots,m-1$. The error $e_m$ satisfies
$$|e_m|\leq\dddd{1}{\lambda_0+Dh^\alpha}\sum_{k=1}^{m-1}\lambda_k |e_{m-k}|+|A_m| h^2,$$
$$|e_m|<\dddd{Ch^{2-\alpha}}{\lambda_0+Dh^\alpha}\sum_{k=1}^{m-1}\lambda_k +A h^2,$$
$$|e_m|<\dddd{C (\lambda_0-\lambda_m)h^{2-\alpha}}{\lambda_0+Dh^\alpha} +A h^2=\llll( \dddd{ \lambda_0- \lambda_m+A\llll(\lambda_0+D h^\alpha \rrrr)h^{\alpha}/C}{\lambda_0+Dh^\alpha} \rrrr)Ch^{2-\alpha}.$$
From \eqref{20_20}
$$\lambda_m>\dddd{L}{m^\alpha}=\dddd{L}{X^\alpha}\llll(\dddd{X}{m} \rrrr)^\alpha>\dddd{L}{X^\alpha}\llll(\dddd{X}{n} \rrrr)^\alpha=\dddd{L}{X^\alpha}h^\alpha.$$
Then
$$\lambda_m - A\llll(\lambda_0+D h^\alpha \rrrr)h^{\alpha}/C>\llll(\dddd{L}{X^\alpha}- \dddd{A\llll(\lambda_0+D \rrrr)}{C}\rrrr)h^\alpha.$$
We have that
$$\dddd{L}{X^\alpha}- \dddd{A\llll(\lambda_0+D \rrrr)}{C}>D,$$
because
$$C>\dddd{A X^\alpha (\lambda_0+D)}{L-D X^\alpha}.$$
\end{proof}
{\bf Example:}	The fractional relaxation equation
\begin{align} \label{22_10}
y^{(\alpha)}(x)+Dy(x)=x^{1-\alpha}E_{1,2-\alpha}(x)+De^x,\quad y(0)=1,
\end{align}
has the solution $y(x)=e^x$. In Table 5 we compute the maximum error and the order of the numerical solutions of equation \eqref{22_10} which use approximations \eqref{3_10}, \eqref{5_30} and \eqref{6_20} for the Caputo derivative when $D=-1$ and $\alpha=0.6$. In Table 6 we compute the error and the order of the numerical solution of equation \eqref{22_10} for the $L1$ approximation \eqref{3_10} and values of $D=-2,-5,-7$ and $\alpha=0.5$.
 The numerical results from Table 6 suggest that  when $D<-5$, numerical solutions  $NS[1]$, $NS[9]$ and $NS[12]$ of equation \eqref{22_10} are not suitable for practical use.
\setlength{\tabcolsep}{0.5em}
{ \renewcommand{\arraystretch}{1.1}
\begin{table}[ht]
	\caption{Maximum error and order of numerical solutions $NS[1]$, $NS[9]$ and $NS[12]$  of equation \eqref{22_10} with $D=-1,\alpha=0.6$  on the interval $[0,1]$.}
	\small
	\centering
  \begin{tabular}{ l | c  c | c  c | c  c }
		\hline
		\hline
		\multirow{2}*{ $\quad \boldsymbol{h}$}  & \multicolumn{2}{c|}{{$\boldsymbol{NS[1]}$}} & \multicolumn{2}{c|}{$\boldsymbol{NS[9]}$}  & \multicolumn{2}{c}{$\boldsymbol{NS[12]}$} \\
		\cline{2-7}
   & $Error$ & $Order$  & $Error$ & $Order$  & $Error$ & $Order$ \\
		\hline \hline
$0.003125$    & $0.0004594$  & $1.3890$  & $0.0005372$  & $1.3826$    & $0.0003854$   & $1.3980$       \\
$0.0015625$   & $0.0001750$  & $1.3927$  & $0.0002052$  & $1.3883$    & $0.0001462$   & $1.3989$       \\
$0.00078125$  & $0.0000665$  & $1.3952$  & $0.0000782$  & $1.3922$    & $0.0000554$   & $1.3994$        \\
$0.000390625$ & $0.0000253$  & $1.3968$  & $0.0000297$  & $1.3949$    & $0.0000210$   & $1.3997$        \\
\hline
  \end{tabular}
	\end{table}
	}
\setlength{\tabcolsep}{0.5em}
{ \renewcommand{\arraystretch}{1.1}
\begin{table}[ht]
	\caption{Maximum error and order of numerical solution $NS[1]$,  of equation \eqref{22_10} with $\alpha=0.5$  on the interval $[0,1]$ and $D=-2,-5,-7$.}
	\small
	\centering
  \begin{tabular}{ l | c  c | c  c | c  c }
		\hline
		\hline
		\multirow{2}*{ $\quad \boldsymbol{h}$}  & \multicolumn{2}{c|}{$\boldsymbol{D=-2}$} & \multicolumn{2}{c|}{$\boldsymbol{D=-5}$}  & \multicolumn{2}{c}{$\boldsymbol{D=-7}$} \\
		\cline{2-7}
   & $Error$ & $Order$  & $Error$ & $Order$  & $Error$ & $Order$ \\
		\hline \hline
$0.003125$    & $0.00275681$  & $1.4795$  & $1.4\times 10^{6}$ & $2.0272$    & $6.7\times 10^{16}$   & $4.6422$       \\
$0.0015625$   & $0.00098584$  & $1.4836$  & $438076.6$         & $1.6668$    & $1.3\times 10^{16}$   & $2.5630$       \\
$0.00078125$  & $0.00035150$  & $1.4878$  & $150198.5$         & $1.5443$    & $3.1\times 10^{15}$   & $1.8610$        \\
$0.000390625$ & $0.00012503$  & $1.4913$  & $52927.21$         & $1.5048$    & $1.0\times 10^{15}$   & $1.6150$        \\
\hline
  \end{tabular}
	\end{table}
	}
\section{Fourier Transform Method for Constructing  Approximations of the Caputo Derivative}
The Fourier transform method is an important method for constructing approximations for fractional derivatives. The generating function of an approximation is directly related to the Fourier transform of the approximation. The Fourier transform method is used by Ding and Li \cite{DingLi2015,DingLi2016}, Lubich \cite{Lubich1986}, Tian et al. \cite{TianZhouDeng2012} for constructing approximations for the fractional derivative. In \cite{Dimitrov2016} we use the Fourier transform method and the series expansion formula of the polylogarithm function  to derive the asymptotic expansion formula \eqref{5_10} for the Riemann sum approximation of the fractional integral at the right endpoint. The  expansion formula for the left endpoint of \eqref{5_10} is obtained from the Euler-Maclaurin formula for the definite integral. In this section we use the  Fourier transform method and integration by parts to obtain approximations for the Caputo derivative of order $1-\alpha,2-\alpha$ and $3-\alpha$.

The Fourier Transform of the function $y$ is defined as
$$\mathcal{F}[y(x)](w)=\hat{y}(w)=\int_{-\infty}^{\infty}e^{i w t}y(t)d t.
$$
The Fourier transform has properties
$$\mathcal{F}[y(x-b)](w)=e^{i w b} \hat{y}(w),\quad \mathcal{F}[y(x)*z(x)](w)= \hat{y}(w) \hat{z}(w),$$
$$\mathcal{F}[D^\alpha y(x)](w)=(-iw)^\alpha \hat{y}(w),\quad \mathcal{F}[I^\alpha y(x)](w)=(-iw)^{-\alpha} \hat{y}(w).$$
\subsection{Expansion Formula}
Let $S_n[y]$ be the Riemann sum of the formal fractional integral $K^{-\alpha}y(x)$.
$$S_n[y]=\dddd{1}{h^\alpha}\sum_{k=1}^{n-1}\dddd{y(x-kh)}{k^{1+\alpha}}.$$
By applying Fourier transform to $S_n[y]$ and letting $n\rightarrow\infty$
$$\mathcal{F}[S_\infty[y]](w)=\dddd{1}{h^\alpha}\sum_{k=1}^{\infty}\dddd{e^{ikhw}}{k^{1+\alpha}}=\dddd{1}{h^\alpha}Li_{1+\alpha}\llll(e^{ikhw} \rrrr).$$
From \eqref{4_10} with $s=1+\alpha$ and $x=e^{ikhw}$ we obtain
\begin{align}\label{24_10}
&Li_{1+\alpha}\llll(e^{i w h}\rrrr)=\Gamma(-\alpha)(-i w h)^{\alpha}+\sum_{m=0}^\infty \dddd{\zeta(1+\alpha-m)}{m!}(i w h)^m,\nonumber\\
\dddd{1}{h^\alpha}&Li_{1+\alpha}\llll(e^{i w h}\rrrr)=\Gamma(-\alpha)(-i w)^{\alpha}+\sum_{m=0}^\infty \dddd{(-1)^m}{m!}\zeta(1+\alpha-m)(-i w)^m h^{m-\alpha}.
\end{align}
Denote by $\mathcal{R}^{(\alpha)}[y]$ and $\mathcal{L}^{(\alpha)}[y]$ the left and right asymptotic sums of
$$S_n[y]-\Gamma(-\alpha)y^{(\alpha)}(x).$$
By applying inverse Fourier transform to \eqref{24_10} we obtain the expansion formula for the right asymptotic sum.
\begin{align*}
\mathcal{R}^{(\alpha)}[y]=\sum_{m=0}^\infty \dddd{\zeta(1+\alpha-m)}{m!}y^{(m)}(x)h^{m-\alpha}.
\end{align*}
The result is summarized in Lemma 11.
	\begin{lem} Let $y^{(m)}(0)=0$ for $m=0,1,\cdots$. Then
	\begin{align*}
\dddd{1}{h^\alpha}\sum_{k=1}^{n-1}\dddd{y(x-kh)}{k^{1+\alpha}}=\Gamma(-\alpha)y^{(\alpha)}(x)+\sum_{m=0}^\infty\dddd{(-1)^m}{m!}\zeta(1+\alpha-m)y^{(m)}(x)h^{m-\alpha}.
\end{align*}
	\end{lem}
	From Lemma 11, we obtain the approximation for the Caputo derivative
	\begin{align}\label{25_10}
\dddd{1}{\Gamma(-\alpha)h^\alpha}\llll(\sum_{k=1}^{n-1}\dddd{y_{n-k}}{k^{1+\alpha}}-\zeta(1+\alpha)y_n \rrrr)=y_n^{(\alpha)}&-\dddd{\zeta(\alpha)}{\Gamma(-\alpha)}y'_n h^{1-\alpha}\\
&+\dddd{\zeta(\alpha-1)}{2\Gamma(-\alpha)}y''_n h^{2-\alpha}+O\llll( h^{3-\alpha} \rrrr).\nonumber
\end{align}
Approximation \eqref{25_10} has accuracy $O\llll( h^{3-\alpha} \rrrr)$ when the function $y(x)$ satisfies $y(0)=y'(0)=y''(0)=0$. Let
$$\omega_0^{(\alpha)}=-\zeta(1+\alpha),\;\omega_k^{(\alpha)}=\dddd{1}{k^{1+\alpha}},\quad (k=1,\cdots,n-1),$$
and
$$\omega_n^{(\alpha)}=-\sum_{k=0}^{n-1}\omega_k^{(\alpha)}=\zeta(1+\alpha)-\sum_{k=1}^{n-1}\dddd{1}{k^{1+\alpha}}.$$
\begin{cor}
\begin{align}\label{25_20}
\mathcal{A}^{\omega}_n[y]=\dddd{1}{\Gamma(-\alpha)h^\alpha}\sum_{k=1}^{n}\omega_k^{(\alpha)} y_{n-k}= y_n^{(\alpha)}+O\llll( h^{1-\alpha}\rrrr).
\end{align}
\end{cor}
Approximation \eqref{25_20} satisfies $\mathcal{A}^{\omega}_n[1]=0$, because $\sum_{k=1}^{n}\omega_k^{(\alpha)}=0$.
\setlength{\tabcolsep}{0.5em}
{ \renewcommand{\arraystretch}{1.1}
\begin{table}[ht]
	\caption{Error and order of numerical solution $NS[34]$ of Equation I with $\alpha=0.25$, Equation II with $\alpha=0.5$ and Equation III with $\alpha=0.75$.}
	\small
	\centering
  \begin{tabular}{ l | c  c | c  c | c  c }
		\hline
		\hline
		\multirow{2}*{ $\quad \boldsymbol{h}$}  & \multicolumn{2}{c|}{{\bf Equation I}} & \multicolumn{2}{c|}{{\bf Equation II}}  & \multicolumn{2}{c}{{\bf Equation III}} \\
		\cline{2-7}
   & $Error$ & $Order$  & $Error$ & $Order$  & $Error$ & $Order$ \\
		\hline \hline
$0.003125$    & $0.0098745$  & $0.7512$  & $0.0298944$   & $0.5069$    & $0.370920$  & $0.2924$       \\
$0.0015625$   & $0.0058683$  & $0.7508$  & $0.0298944$   & $0.5051$    & $0.304271$  & $0.2858$       \\
$0.00078125$  & $0.0034881$  & $0.7505$  & $0.0148565$   & $0.5037$    & $0.250614$  & $0.2799$        \\
$0.000390625$ & $0.0020736$  & $0.7503$  & $0.0104857$   & $0.5027$    & $0.207125$  & $0.2750$        \\
\hline
  \end{tabular}
	\end{table}
	}
	Denote by $\mathcal{L}[y]=\mathcal{L}^{(1)}[y]$ the left asymptotic sum of
	$$h\sum_{k=1}^{n-1}y(kh)-\int_0^x y(t) d t.$$
From the	Euler-Maclaurin formula
$$h\sum_{k=1}^{n-1}y(kh)=\int_0^x y(t) d t-(y(0)+y(x))\dddd{h}{2}+\sum_{m=1}^\infty \dddd{B_{2m}h^{2m}}{(2m)!}\llll(y^{(2m-1)}(x)-y^{(2m-1)}(0)\rrrr)$$
we obtain
$$\mathcal{L}[y]=-\dddd{y(0) h}{2}-\sum_{m=1}^\infty \dddd{B_{2m}}{(2m)!}y^{(2m-1)}(0)h^{2m}.$$
Now we use the Euler-Maclaurin formula and integration
	by parts to derive the formula for $\mathcal{L}^{(\alpha)}[y]$.
			\begin{align*}
\Gamma(1-\alpha)y^{(\alpha)}(x)=\int_0^x \dddd{y'(t)}{(x-t)^\alpha}dt=\int_0^{x/2} \dddd{y'(t)}{(x-t)^\alpha}dt+\int_{x/2}^x \dddd{y'(t)}{(x-t)^\alpha}dt.
\end{align*}
	We have that
\begin{align*}
	\int_0^{x/2} \dddd{y'(t)}{(x-t)^\alpha}dt&=\llll.(x-t)^{-\alpha} y(t) \rrrr|_0^{x/2}-\int_0^{x/2} y(t) d\llll[(x-t)^{-\alpha} \rrrr]\\
	&=\llll(\dddd{x}{2}\rrrr)^{-\alpha} y\llll(\dddd{x}{2}\rrrr)-x^{-\alpha} y(0)-\alpha \int_0^{x/2} \dddd{y(t)}{(x-t)^{1+\alpha}} dt.
\end{align*}
	The gamma function satisfies $\Gamma(1-\alpha)=-\alpha \Gamma(-\alpha)$. Then
			\begin{align}\label{26_10}
\Gamma(-\alpha)y^{(\alpha)}(x)=\dddd{ y(0)}{\alpha x^{\alpha}}+ \int_0^{x/2} &\dddd{y(t)}{(x-t)^{1+\alpha}} dt\\
&-\dddd{1}{\alpha}\llll(\llll(\dddd{x}{2}\rrrr)^{-\alpha} y\llll(\dddd{x}{2}\rrrr)+\int_{x/2}^x \dddd{y'(t)}{(x-t)^\alpha}dt\rrrr).\nonumber
\end{align}
Let
$$z(t)=\dddd{y(t)}{(x-t)^{1+\alpha}}=y(t)Z(t),$$
where $Z(t)=(x-t)^{-1-\alpha}$. From \eqref{26_10} we have that
$$\mathcal{L}^{(\alpha)}[y]=\mathcal{L}[z]+y(0)x^{-\alpha}/\alpha .$$
The function $Z(t)$ has derivatives
$$Z^{(k)}(t)=\dddd{\Gamma(1+\alpha+k)}{\Gamma(1+\alpha)}\dddd{1}{(x-t)^{1+\alpha+k}}.$$
From the Leibnitz's rule
$$z^{(2m-1)}(0)=\sum_{k=0}^{2m-1}\binom{2m-1}{k}y^{(2m-1-k)}(0)Z^{(k)}(0).$$
Then
$$\mathcal{L}[z]=-\dddd{z(0) h}{2}-\sum_{m=1}^\infty \dddd{B_{2m}}{(2m)!}z^{(2m-1)}(0)h^{2m},$$
\begin{align}\label{27_10}
\mathcal{L}[z]=&-\dddd{y(0) h}{2 x^{1+\alpha}}\\
&-\sum_{m=1}^\infty \dddd{B_{2m}}{(2m)!}\llll( \sum_{k=0}^{2m-1}\binom{2m-1}{k}\dddd{\Gamma(1+\alpha+k)}{\Gamma(1+\alpha)}\dddd{y^{(2m-1-k)}(0)}{x^{1+\alpha+k}}\rrrr)h^{2m}.\nonumber
\end{align}
From \eqref{27_10} and Lemma 11 we obtain \eqref{27_20}.
	\begin{lem} Asymptotic expansion formula
	\begin{align}\label{27_20}
\dddd{1}{h^\alpha}\sum_{k=1}^{n-1}\dddd{y(x-kh)}{k^{1+\alpha}}=&\Gamma(-\alpha)y^{(\alpha)}(x)+\dddd{ y(0)}{\alpha x^{\alpha}}-\dddd{y(0) h}{2 x^{1+\alpha}}\\
&+\sum_{m=0}^\infty\dddd{(-1)^m}{m!}\zeta(1+\alpha-m)y^{(m)}(x)h^{m-\alpha}\nonumber\\
-\sum_{m=1}^\infty \dddd{B_{2m}}{(2m)!}&\llll( \sum_{k=0}^{2m-1}\binom{2m-1}{k}\dddd{\Gamma(1+\alpha+k)}{\Gamma(1+\alpha)}\dddd{y^{(2m-1-k)}(0)}{x^{1+\alpha+k}}\rrrr)h^{2m}.\nonumber
\end{align}
\end{lem}
From \eqref{27_20} we obtain the fourth-order approximation
\begin{align}\label{27_30}
\Gamma(-\alpha)y^{(\alpha)}(x)=&\dddd{1}{h^\alpha}\sum_{k=1}^{n-1}\dddd{y(x-kh)}{k^{1+\alpha}}-\zeta(1+\alpha)\dddd{y(x)}{h^\alpha}+\dddd{y(0)}{\alpha x^\alpha}
+\dddd{y(0)h}{2 x^{1+\alpha}}\\
+\zeta(\alpha)&y'(x)h^{1-\alpha}-\dddd{\zeta(\alpha-1)}{2}y''(x)h^{2-\alpha}+\dddd{\zeta(\alpha-2)}{6}y'''(x)h^{3-\alpha}\nonumber\\
-&\dddd{\zeta(\alpha-3)}{24}y^{(4)}(x)h^{4-\alpha}+\dddd{ x y'(0)+(1+\alpha)y(0)}{12x^{2+\alpha}}h^2+O\llll( h^4 \rrrr).\nonumber
\end{align}
{\renewcommand{\arraystretch}{1.1}
\begin{table}[ht]
	\caption{Maximum error and order of approximation \eqref{27_30} for the functions $\tan^{-1} t,x=1,\; \ln (t+1),x=2$ and $\zeta (t+2),x=3$ with $\alpha=0.4$.}
	\small
	\centering
  \begin{tabular}{ l | c  c | c  c | c  c }
		\hline
		\hline
		\multirow{2}*{ $\quad \boldsymbol{h}$}  & \multicolumn{2}{c|}{$\boldsymbol{\tan^{-1} t}$} & \multicolumn{2}{c|}{$\boldsymbol{\ln (t+1)}$}  & \multicolumn{2}{c}{$\boldsymbol{\zeta (t+2)}$} \\
		\cline{2-7}
   & $Error$ & $Order$  & $Error$ & $Order$  & $Error$ & $Order$ \\
		\hline \hline
$0.025$    &  $4.4\times 10^{-9}$  & $3.999$  & $5.0\times 10^{-10}$  & $3.999$    & $4.2\times 10^{-10}$   & $3.999$       \\
$0.0125$   &  $2.7\times 10^{-10}$ & $3.999$  & $3.1\times 10^{-11}$  & $4.000$    & $2.6\times 10^{-11}$   & $3.999$       \\
$0.00625$  &  $1.7\times 10^{-11}$ & $3.999$  & $2.0\times 10^{-12}$  & $3.969$    & $1.6\times 10^{-12}$   & $4.003$        \\
$0.003125$ &  $1.1\times 10^{-12}$ & $3.961$  & $1.5\times 10^{-13}$  & $3.734$    & $1.2\times 10^{-13}$   & $3.750$        \\
\hline
  \end{tabular}
	\end{table}
	}
\subsection{Approximation for the Caputo Derivative of Order $\boldsymbol{2-\alpha}$}
	By approximating  $y'_n$ in \eqref{25_10} using first-order backward difference approximation
	$y'_n=(y_n-y_{n-1})/h+O(h)$ we obtain the approximation
\begin{align}\label{28_10}
\mathcal{A}_n^{\bar{\sigma}}[y]=\dddd{1}{\Gamma(-\alpha)h^\alpha}\sum_{k=0}^{n}\bar{\sigma}_k^{(\alpha)} y_{n-k}= y^{(\alpha)}_n+O\llll(h^{2-\alpha}\rrrr),
\end{align}
where
$$\bar{\sigma}_0^{(\alpha)}=\zeta(\alpha)-\zeta(1+\alpha),\;\bar{\sigma}_1^{(\alpha)}=1-\zeta(\alpha),$$
$$\bar{\sigma}_k^{(\alpha)}=\dddd{1}{k^{1+\alpha}}\quad (k=2,\cdots,n-1),\quad \bar{\sigma}_{n}^{(\alpha)}=\zeta(1+\alpha)-\sum_{k=1}^{n-1}\dddd{1}{k^{1+\alpha}}.$$
Approximation \eqref{28_10} satisfies $\mathcal{A}^{\bar{\sigma}}_n[1]=0$. The accuracy of approximation \eqref{28_10} is $O\llll(h^{2-\alpha}\rrrr)$ when the function $y$ satisfies $y'(0)=0$.
Denote
$$W_n[\alpha]=n S_n[\alpha+1]-S_n[\alpha]+\dddd{n^{1-\alpha}}{\alpha(1-\alpha)}.$$
\begin{clm} Let $y(x)=x$. Then
\begin{align*}
\mathcal{A}^{\bar{\sigma}}_n[x]-y^{(\alpha)}(x)=\dddd{W_n[\alpha]h^{1-\alpha}}{\Gamma(-\alpha)}.
\end{align*}
\end{clm}
\begin{proof}
\begin{align*}
\Gamma(-\alpha&)h^\alpha\mathcal{A}^{\bar{\sigma}}_n[x]=\sum_{k=0}^{n}\bar{\sigma}_k^{(\alpha)} (x-kh)=-h\sum_{k=1}^{n}k\bar{\sigma}_k^{(\alpha)}\\
&=-h\llll(\sum_{k=1}^{n-1}k\bar{\sigma}_k^{(\alpha)}+n \bar{\sigma}_n^{(\alpha)} \rrrr)=h\llll(\zeta(\alpha)-n\omega_n^{(\alpha)}-\sum_{k=1}^{n-1}k\omega_k^{(\alpha)}\rrrr),
\end{align*}
$$\mathcal{A}^{\bar{\sigma}}_n[x]=\dddd{h^{1-\alpha}}{\Gamma(-\alpha)}\llll(\llll(\zeta(\alpha)-\sum_{k=1}^{n-1}\dddd{1}{k^{\alpha}}  \rrrr)-n\llll(\zeta(1+\alpha)-\sum_{k=1}^{n-1}\dddd{1}{k^{1+\alpha}} \rrrr)\rrrr).$$
Then
\begin{align*}
\mathcal{A}^{\bar{\sigma}}_n[y]-y^{(\alpha)}(x)&=\dddd{h^{1-\alpha}}{\Gamma(-\alpha)}(n S_n(\alpha+1)-S_n(\alpha))-\dddd{x^{1-\alpha}}{\Gamma(2-\alpha)}\\
&=\dddd{h^{1-\alpha}}{\Gamma(-\alpha)}\llll(n S_n(\alpha+1)-S_n(\alpha)+\dddd{n^{1-\alpha}}{\alpha(1-\alpha)}\rrrr).
\end{align*}
\end{proof}
Represent the function $y(x)$ as
$$y(x)=y(x)-y(0)-y'(0)x+y(0)+y'(0)x=z(x)+y(0)+y'(0)x.$$
The function $z(x)=y(x)-y(0)-y'(0)x$ satisfies $z(0)=z'(0)=0$.
$$\mathcal{A}^{\bar{\sigma}}_n[z]=z^{(\alpha)}(x)+O\llll( h^{2-\alpha} \rrrr)=y^{(\alpha)}(x)-y'(0)\dddd{x^{1-\alpha}}{\Gamma(2-\alpha)}+O\llll( h^{2-\alpha} \rrrr).$$
Then
$$\mathcal{A}^{\bar{\sigma}}_n[y]=\mathcal{A}^{\bar{\sigma}}_n[z]+y(0)\mathcal{A}^{\bar{\sigma}}_n[1]+y'(0)\mathcal{A}^{\bar{\sigma}}_n[x],$$
$$\mathcal{A}^{\bar{\sigma}}_n[y]=y^{(\alpha)}(x)+y'(0)\mathcal{A}^{\bar{\sigma}}_n[x]-y'(0)\dddd{x^{1-\alpha}}{\Gamma(2-\alpha)}+O\llll( h^{2-\alpha} \rrrr).$$
From Claim 14
$$\dddd{1}{\Gamma(-\alpha)h^\alpha}\sum_{k=0}^{n}\bar{\sigma}_k^{(\alpha)} y_{n-k}= y^{(\alpha)}_n+\llll(\dddd{y'(0) W_n[\alpha]}{\Gamma(-\alpha)}\rrrr)h^{1-\alpha}+O\llll(h^{2-\alpha}  \rrrr)$$
By approximating $y'_0$ using first-order forward difference approximation
$$y'_0=\dddd{y_1-y_0}{h}+O(h)$$
we obtain the approximation for the Caputo derivative of order $2-\alpha$
\begin{align}\label{29_10}
\dddd{1}{\Gamma(-\alpha)h^\alpha}\sum_{k=0}^{n}\sigma_k^{(\alpha)} y_{n-k}= y_n^{(\alpha)}+O\llll(h^{2-\alpha}\rrrr),
\end{align}
where $\sigma_0^{(\alpha)}=\zeta(\alpha)-\zeta(1+\alpha),\;\sigma_1^{(\alpha)}=1-\zeta(\alpha),$
$$\sigma_k^{(\alpha)}=\dddd{1}{k^{1+\alpha}}\quad (k=2,\cdots,n-2),$$
$$\sigma_{n-1}^{(\alpha)}=\dddd{1}{(n-1)^{1+\alpha}}-\dddd{n^{1-\alpha}}{\alpha(1-\alpha)}-n S_n[1+\alpha]+S_n[\alpha],$$
$$\sigma_{n}^{(\alpha)}=(n-1)S_n[1+\alpha]+S_n[\alpha]+\dddd{n^{1-\alpha}}{\alpha(1-\alpha)}.$$
When $n=2$ approximation \eqref{29_10} has weights $\sigma_0^{(\alpha)}=\zeta(\alpha)-\zeta(1+\alpha)$ and
$$\sigma_1^{(\alpha)}=2\zeta(1+\alpha)-2\zeta(\alpha)-\dddd{2^{1-\alpha}}{\alpha(1-\alpha)},\; \sigma_2^{(\alpha)}=\zeta(\alpha)-\zeta(1+\alpha)+\dddd{2^{1-\alpha}}{\alpha(1-\alpha)}.$$
From the formula for sum of powers \eqref{14_05}
$$S_n[\alpha]=\dddd{n^{1-\alpha}}{1-\alpha}\sum_{m=0}^{\infty}\binom{1-\alpha}{m}\dddd{B_m}{n^m},$$
The Bernoulli number $B_1=-1/2$ and $B_{2m+1}=0$ for $m\geq 1$. Then
\begin{align}\label{30_10}
\dddd{n^{1-\alpha}}{1-\alpha}\sum_{m=1}^{\infty}\binom{1-\alpha}{2m}\dddd{B_{2m}}{n^{2m}}=S_n[\alpha]-\dddd{n^{1-\alpha}}{1-\alpha}+\dddd{1}{2n^\alpha}.
\end{align}
From \eqref{30_10}
$$S_n[\alpha]=\dddd{n^{1-\alpha}}{1-\alpha}-\dddd{1}{2n^\alpha}+O\llll( \dddd{1}{n^{1+\alpha}}\rrrr),\; S_n[\alpha+1]=-\dddd{n^{-\alpha}}{\alpha}-\dddd{1}{2n^{1+\alpha}}+O\llll( \dddd{1}{n^{2+\alpha}}\rrrr).
$$
Then
$$\tilde{\sigma}_n^{(\alpha)}= (1-n)\llll(\dddd{1}{\alpha n^\alpha} +\dddd{1}{2 n^{1+\alpha}}\rrrr)-\llll(\dddd{n^{1-\alpha}}{1-\alpha}-\dddd{1}{2n^\alpha} \rrrr)+\dddd{n^{1-\alpha}}{\alpha(1-\alpha)}+O\llll(\dddd{1}{n^{1+\alpha}} \rrrr),$$
$$\sigma_n^{(\alpha)}= \dddd{1}{\alpha n^\alpha} +\llll(\dddd{n^{1-\alpha}}{\alpha(1-\alpha)}-\dddd{n^{1-\alpha}}{\alpha} -\dddd{n^{1-\alpha}}{1-\alpha}\rrrr)+O\llll(\dddd{1}{n^{1+\alpha}} \rrrr)=\dddd{1}{\alpha n^\alpha}+O\llll(\dddd{1}{n^{1+\alpha}} \rrrr).$$
The weight $\sigma_n^{(\alpha)}$ of approximation \eqref{29_10} is approximately $1/(\alpha n^\alpha)$. Approximation \eqref{29_10} has order $2-\alpha$ and its weights satisfy \eqref{3_20}. In Table 9 we compute the error and the order of the numerical solution of Equation I, Equation II and Equation III which uses approximation \eqref{29_10} for the Caputo derivative.
In all experiments in Table 1, Table 3, Table 5 and Table 9 the errors of numerical solution $NS[12]$ are smaller than the errors of numerical solutions $NS[1]$ and $NS[9]$.
{ \renewcommand{\arraystretch}{1.1}
\begin{table}[ht]
	\caption{Error and order of numerical solution $NS[40]$ of Equation I with $\alpha=0.25$, Equation II with $\alpha=0.5$ and Equation III with $\alpha=0.75$.}
	\small
	\centering
  \begin{tabular}{ l | c  c | c  c | c  c }
		\hline
		\hline
		\multirow{2}*{ $\quad \boldsymbol{h}$}  & \multicolumn{2}{c|}{{\bf Equation I}} & \multicolumn{2}{c|}{{\bf Equation II}}  & \multicolumn{2}{c}{{\bf Equation III}} \\
		\cline{2-7}

   & $Error$ & $Order$  & $Error$ & $Order$  & $Error$ & $Order$ \\
		\hline \hline
$0.003125$    & $0.0000263$          & $1.7472$  & $0.0000395122$        & $1.4982$    & $0.0022174$  & $1.2479$       \\
$0.0015625$   & $7.8\times 10^{-6}$  & $1.7486$  & $0.0000139784$        & $1.4991$    & $0.0009328$  & $1.2492$       \\
$0.00078125$  & $2.3\times 10^{-6}$  & $1.7493$  & $4.9\times 10^{-6}$   & $1.4995$    & $0.0003923$  & $1.2497$        \\
$0.000390625$ & $6.9\times 10^{-7}$  & $1.7497$  & $1.7\times 10^{-6}$   & $1.4998$    & $0.0001649$  & $1.2499$        \\
\hline
  \end{tabular}
	\end{table}
	}
\subsection{Approximation for the Caputo Derivative of Order $\boldsymbol{3-\alpha}$}
From \eqref{27_20}, approximation \eqref{25_20} has asymptotic expansion
	\begin{align}\label{31_10}
\dddd{1}{h^\alpha}\sum_{k=0}^{n}\omega_k^{(\alpha)} y(x-kh&)=\Gamma(-\alpha)y^{(\alpha)}(x)+\sum_{m=1}^\infty\dddd{(-1)^m}{m!}\zeta(1+\alpha-m)y^{(m)}(x)h^{m-\alpha}\nonumber\\
-\sum_{m=1}^\infty \dddd{B_{2m}}{(2m)!}&\llll( \sum_{k=0}^{2m-2}\binom{2m-1}{k}\dddd{\Gamma(1+\alpha+k)}{\Gamma(1+\alpha)}\dddd{y^{(2m-1-k)}(0)}{x^{1+\alpha+k}}\rrrr)h^{2m}
\end{align}
In order to construct approximations for the Caputo derivative we express expansion formula \eqref{31_10} in the following form
	\begin{align*}
\dddd{1}{h^\alpha}\sum_{k=0}^{n}\omega_k^{(\alpha)} y(x-k h)=&\Gamma(-\alpha)y^{(\alpha)}(x)+\sum_{m=1}^\infty K_m(\alpha)y^{(m)}(0) h^{m-\alpha}\\
&+\sum_{m=1}^\infty\dddd{(-1)^m}{m!}\zeta(1+\alpha-m)y^{(m)}(x)h^{m-\alpha}.
\end{align*}
In the previous section we showed that
$$K_1(\alpha)=n S_n[1+\alpha]-S_n[\alpha]+\dddd{n^{1-\alpha}}{\alpha(1-\alpha)}.$$
In this section we obtain the formula \eqref{33_10} for the coefficient $K_2(\alpha)$ and an approximation \eqref{34_10} for the Caputo derivative of order $3-\alpha$. By changing the order of summation in \eqref{31_10}
	\begin{align*}
\dddd{1}{h^\alpha}\sum_{k=0}^{n}\omega_k^{(\alpha)} y(x-kh)=&\Gamma(-\alpha)y^{(\alpha)}(x)+\sum_{m=1}^\infty\dddd{(-1)^m}{m!}\zeta(1+\alpha-m)y^{(m)}(x)h^{m-\alpha}\\
-\sum_{p=1}^\infty &\llll( \sum_{m=\left\lceil  p/2\right\rceil}^{\infty}\binom{2m-1}{p}\dddd{\Gamma(2m+\alpha-p)}{\Gamma(1+\alpha)x^{2m+\alpha-p}}\dddd{B_{2m}}{(2m)!}h^{2m}\rrrr)y^{(p)}(0).
\end{align*}
We have that $x^{2m+\alpha-p}=n^{2m+\alpha-p}h^{2m+\alpha-p}$. Then
$$K_p(\alpha)= -\sum_{m=\left\lceil  p/2\right\rceil}^{\infty}\binom{2m-1}{p}\dddd{\Gamma(2m+\alpha-p)}{\Gamma(1+\alpha)n^{2m+\alpha-p}}\dddd{B_{2m}}{(2m)!}.$$
When $p=2$ we obtain
$$K_2(\alpha)=-\dddd{n^{2-\alpha}}{2} \sum_{m=1}^{\infty}(2m-1)(2m-2)\dddd{\Gamma(2m+\alpha-2)}{\Gamma(1+\alpha)(2m)!}\dddd{B_{2m}}{n^{2m}}.$$
From the identity
$$(2m-1)(2m-2)=(2m-2+\alpha)(2m-1+\alpha)-2\alpha(2m-2+\alpha)+\alpha(\alpha-1)$$
we obtain
\begin{align*}
K_2(\alpha)=&-\dddd{n^{2-\alpha}}{2} \sum_{m=1}^{\infty}\dddd{(1+\alpha)(2+\alpha)\cdots (2m-1+\alpha)}{(2m)!}\dddd{B_{2m}}{n^{2m}}\\
&+n^{2-\alpha} \sum_{m=1}^{\infty}\dddd{\alpha(1+\alpha)(2+\alpha)\cdots (2m-2+\alpha)}{(2m)!}\dddd{B_{2m}}{n^{2m}}\\
&-\dddd{n^{2-\alpha}}{2} \sum_{m=1}^{\infty}\dddd{(\alpha-1)a(1+\alpha)(2+\alpha)\cdots (2m-3+\alpha)}{(2m)!}\dddd{B_{2m}}{n^{2m}},
\end{align*}
\begin{align*}
K_2(\alpha)=&- \dddd{n^{2-\alpha}}{2\alpha}\sum_{m=1}^{\infty}\dddd{\alpha(1+\alpha)(2+\alpha)\cdots (2m-1+\alpha)}{(2m)!}\dddd{B_{2m}}{n^{2m}}\\
&+\dddd{n^{2-\alpha}}{\alpha-1} \sum_{m=1}^{\infty}\dddd{(\alpha-1)\alpha(1+\alpha)(2+\alpha)\cdots (2m-2+\alpha)}{(2m)!}\dddd{B_{2m}}{n^{2m}}\\
&-\dddd{n^{2-\alpha}}{2(\alpha-2)} \sum_{m=1}^{\infty}\dddd{(\alpha-2)(\alpha-1)a(1+\alpha)(2+\alpha)\cdots (2m-3+\alpha)}{(2m)!}\dddd{B_{2m}}{n^{2m}},
\end{align*}
\begin{align*}
K_2(\alpha)=&\dddd{n^{2}}{2}\llll[ \dddd{n^{-\alpha}}{-\alpha}\sum_{m=1}^{\infty}\binom{-\alpha}{2m}\dddd{B_{2m}}{n^{2m}}\rrrr]-n\llll[\dddd{n^{1-\alpha}}{1-\alpha} \sum_{m=1}^{\infty}\binom{1-\alpha}{2m}\dddd{B_{2m}}{n^{2m}}\rrrr]\\
&+\dddd{1}{2}\llll[\dddd{n^{2-\alpha}}{2-\alpha} \sum_{m=1}^{\infty}\binom{2-\alpha}{2m}\dddd{B_{2m}}{n^{2m}}\rrrr].
\end{align*}
From \eqref{30_10}
$$\dddd{n^{-\alpha}}{-\alpha}\sum_{m=1}^{\infty}\binom{-\alpha}{2m}\dddd{B_{2m}}{n^{2m}}=S_n[\alpha+1]+\dddd{n^{-\alpha}}{\alpha}+\dddd{1}{2n^{1+\alpha}},$$
$$\dddd{n^{2-\alpha}}{2-\alpha}\sum_{m=1}^{\infty}\binom{2-\alpha}{2m}\dddd{B_{2m}}{n^{2m}}=S_n[\alpha-1]-\dddd{n^{2-\alpha}}{2-\alpha}+\dddd{1}{2n^{\alpha-1}}.$$
Then
\begin{align*}
K_2(\alpha)=&\dddd{n^{2}}{2}\llll[S_n[\alpha+1]+\dddd{n^{-\alpha}}{\alpha}+\dddd{1}{2n^{1+\alpha}}\rrrr]-n\llll[S_n[\alpha]-\dddd{n^{1-\alpha}}{1-\alpha}+\dddd{1}{2n^\alpha}\rrrr]\\
&+\dddd{1}{2}\llll[S_n[\alpha-1]-\dddd{n^{2-\alpha}}{2-\alpha}+\dddd{1}{2n^{\alpha-1}}\rrrr],
\end{align*}
\begin{align}\label{33_10}
K_2(\alpha)=\dddd{n^{2}}{2}S_n[\alpha+1]-n S_n[\alpha]+\dddd{1}{2}S_n[\alpha-1]+\dddd{n^{2-\alpha}}{\alpha(\alpha-1)(\alpha-2)}.
\end{align}
Approximation \eqref{25_20} has expansion of order $3-\alpha$
	\begin{align}\label{33_20}
\dddd{1}{h^\alpha}\sum_{k=0}^{n}\omega_k^{(\alpha)} y_{n-k}=\Gamma(-\alpha)y^{(\alpha)}_n+&\llll( K_1(\alpha) y'_0 -\zeta(\alpha) y'_n\rrrr) h^{1-\alpha}\\
&+\llll( K_2(\alpha) y''_0 -\dddd{1}{2}\zeta(\alpha) y''_n\rrrr) h^{2-\alpha}+O\llll(h^{3-\alpha} \rrrr).\nonumber
\end{align}
By  approximating $y'_0,y''_0,y'_n,y''_n$ in \eqref{33_20} using the approximations
$$y'_0=\dddd{1}{h}\llll(-\dddd{3}{2}y_0+2y_1-\dddd{1}{2}y_2 \rrrr)+O\llll(h^2\rrrr),\; y''_0=\dddd{1}{h^2}\llll(y_0-2y_1+y_2 \rrrr)+O\llll(h^2\rrrr),$$
$$y'_n=\dddd{1}{h}\llll(\dddd{3}{2}y_n-2y_{n-1}+\dddd{1}{2}y_{n-2}\rrrr)+O\llll(h^2\rrrr),\; y''_n=\dddd{1}{h^2}\llll(y_n-2y_{n-1}+y_{n-2}\rrrr)+O\llll(h\rrrr),$$
we obtain the approximation for the Caputo derivative of order $3-\alpha$
\begin{align}\label{34_10}
\dddd{1}{\Gamma(-\alpha)h^\alpha}\sum_{k=0}^{n}\delta_k^{(\alpha)} y_{n-k}= y_n^{(\alpha)}+O\llll(h^{3-\alpha}\rrrr),
\end{align}
where
$$\delta_0^{(\alpha)}=-\zeta(1+\alpha)+\dddd{3}{2}\zeta(\alpha)-\dddd{1}{2}\zeta(\alpha-1),\;\delta_1^{(\alpha)}=1-2\zeta(\alpha)+\zeta(\alpha-1),$$
$$\delta_2^{(\alpha)}=\dddd{1}{2^{1+\alpha}}+\dddd{1}{2}\zeta(\alpha)-\dddd{1}{2}\zeta(\alpha-1),\;\delta_k^{(\alpha)}=\dddd{1}{k^{1+\alpha}}\quad (k=3,\cdots,n-3),$$
$$\delta_{n-2}^{(\alpha)}=\omega_{n-2}^{(\alpha)}+\dddd{1}{2}K_1(\alpha)-K_2(\alpha),\;\delta_{n-1}^{(\alpha)}=\omega_{n-1}^{(\alpha)}-2K_1(\alpha)+2K_2(\alpha),$$
$$\delta_n^{(\alpha)}=\omega_n^{(\alpha)}+\dddd{3}{2}K_1(\alpha)-K_2(\alpha).$$
When $n=2$, approximation \eqref{34_10} has weights
$$
\delta_0^{(\alpha)}=\dddd{ (\alpha+2)}{2^{\alpha}(2-\alpha) (\alpha-1)},\; \delta_1^{(\alpha)}=\dddd{4}{2^\alpha (2-\alpha) (1-\alpha)},\; \delta_2^{(\alpha)}=\dddd{ 2-3 \alpha}{2^{\alpha}(2-\alpha) (1-\alpha) \alpha}.$$
When $n=3$: $\delta_0^{(\alpha)}=-\frac{1}{2}\zeta (\alpha-1)+\frac{3 }{2}\zeta (\alpha)-\zeta (\alpha+1),$
$$\delta_1^{(\alpha)}=\dddd{3 }{2}\zeta (\alpha-1)+3 \zeta (\alpha+1)-\dddd{9 }{2}\zeta (\alpha)-\dddd{3^{1-\alpha} (\alpha+4)}{2 \alpha (1-\alpha) (2-\alpha)},$$
$$ \delta_2^{(\alpha)}=-\dddd{3}{2}\zeta (\alpha-1)+\dddd{9 }{2}\zeta (\alpha)-3 \zeta (\alpha+1)+\dddd{2\times 3^{1-\alpha} (\alpha+1)}{\alpha(1-\alpha) (2-\alpha)},$$
$$\delta_3^{(\alpha)}=\dddd{1}{2}\zeta (\alpha-1)+\zeta (\alpha+1)-\dddd{3}{2}\zeta (\alpha)-\dddd{3^{2-\alpha}}{2 (1-\alpha) (2-\alpha)}.$$
In the previous sections computed the numerical solutions of the fractional relaxation equation of order $1-\alpha,2-\alpha$ and $2$ using the second-order approximation \eqref{19_20} for the value $y(h)$. In order to obtain a third-order approximation for  $y(h)$ we need to compute the values of $y'(0)$ and $y''(0)$.
By applying fractional differentiation of order $1-\alpha$ to equation \eqref{19_10} we obtain
\begin{align}\label{34_20}
y'(x)+y^{(1-\alpha)}(x)=F^{(1-\alpha)}(x).
\end{align}
Then $y'(0)=F^{(1-\alpha)}(0)$. In \cite{Dimitrov2016} we showed that
$$\dddd{d}{dx}y^{(1-\alpha)}(x)=y^{(2-\alpha)}(x)+\dddd{y'(0)x^{\alpha-1}}{\Gamma(\alpha)}.$$
By differentiating \eqref{34_20} we obtain
$$y''(x)+\dddd{d}{dx}y^{(1-\alpha)}(x)=\dddd{d}{dx}F^{(1-\alpha)}(x),$$
$$y''(x)+y^{(2-\alpha)}(x)+\dddd{y'(0)x^{\alpha-1}}{\Gamma(\alpha)}=\dddd{d}{dx}F^{(1-\alpha)}(x).$$
When the solution of equation \eqref{19_10} has a bounded second derivative, the value of $y''(0)$ is computed with the formula
$$y''(0)=\lim_{x\rightarrow 0}\llll(\dddd{d}{dx}F^{(1-\alpha)}(x)-\dddd{y'(0)x^{\alpha-1}}{\Gamma(\alpha)}\rrrr).$$
Now we compute the values of the derivatives $y'(0),y''(0)$ of the exact solutions of Equation I, Equation II and Equation III.
$$F_1(x)=\sum_{k=0}^4 x^k+\sum_{k=1}^4 \dddd{k! x^{k-\alpha}}{\Gamma(k+1-\alpha)},$$
$$F_1^{(1-\alpha)}(x)=\sum_{k=0}^3 \dddd{x^{k+\alpha}}{\Gamma(k+1-\alpha)}+\sum_{k=0}^3 (k+1) x^k.$$
Then $y'(0)=F_1^{(1-\alpha)}(0)=1$.
$$\dddd{d }{d x}F_1^{(1-\alpha)}(x)=\sum_{k=0}^3 \dddd{x^{k+\alpha-1}}{\Gamma(k-\alpha)}+\sum_{k=1}^3 k(k+1) x^{k-1}.$$
Then
\begin{align*}
y''(0)=\lim_{x\rightarrow 0}\Bigg(\dddd{d}{dx}&F_1^{(1-\alpha)}(x)-\dddd{x^{\alpha-1}}{\Gamma(\alpha)}\Bigg)\\
&=\lim_{x\rightarrow 0}\sum_{k=1}^3 \llll(\dddd{x^{k+\alpha-1}}{\Gamma(k-\alpha)}+ k(k+1) x^{k-1}\rrrr)=2
\end{align*}
The exact solution of Equation I satisfies $y'(0)=1$ and  $y''(0)=2$.
$$F_{2}(x)=e^x+x^{1-\alpha}E_{1,2-\alpha}(x)=\sum_{k=0}^\infty \dddd{x^k}{k!}+\sum_{k=1}^4 \dddd{ x^{k+1-\alpha}}{\Gamma(k+2-\alpha)},$$
$$F_{2}^{(1-\alpha)}(x)=\sum_{k=1}^\infty \dddd{x^{k-1+\alpha}}{\Gamma(k+\alpha)}+\sum_{k=0}^\infty \dddd{x^k}{k!}=\sum_{k=1}^\infty \dddd{x^{k-1+\alpha}}{\Gamma(k+\alpha)}+e^x.$$
Then $y'(0)=F_{2}^{(1-\alpha)}(0)=1$.
$$\dddd{d }{dx}F_{2}^{(1-\alpha)}(x)=\sum_{k=1}^\infty \dddd{x^{k-2+\alpha}}{\Gamma(k+\alpha-1)}+e^x,$$
$$y''(0)=\lim_{x\rightarrow 0}\llll(\dddd{d}{dx}F_{2}^{(1-\alpha)}(x)-\dddd{x^{\alpha-1}}{\Gamma(\alpha)}\rrrr)=\lim_{x\rightarrow 0}\llll(\sum_{k=2}^\infty \dddd{x^{k-2+\alpha}}{\Gamma(k+\alpha-1)}+e^x\rrrr)=1.$$
The exact solution of Equation II satisfies $y'(0)=y''(0)=1$.
\begin{align*}
F_{3}(x)=\cos (2\pi x)+i \pi x^{1-\alpha}
\llll(E_{1,2-\alpha}(2\pi i x)-E_{1,2-\alpha}(-2\pi i x)\rrrr),
\end{align*}
$$F_{3}(x)=\sum_{k=0}^\infty \dddd{\llll(-4\pi^2 \rrrr)^k x^{2k}}{(2k)!}+\sum_{k=1}^\infty \dddd{\llll(-4\pi^2 \rrrr)^k x^{2k-\alpha}}{\Gamma(2k+1-\alpha)},$$
$$F_{3}^{(1-\alpha)}(x)=\sum_{k=1}^\infty \llll(-4\pi^2 \rrrr)^k\llll( \dddd{ x^{2k-1+\alpha}}{\Gamma(2k+\alpha)}+\dddd{ x^{2k-1}}{(2k-1)!} \rrrr).$$
Then $y'(0)=F_{3}^{(1-\alpha)}(0)=0$.
$$\dddd{d }{dx}F_{3}^{(1-\alpha)}(x)=\sum_{k=1}^\infty \llll(-4\pi^2 \rrrr)^k\llll( \dddd{ x^{2k-2+\alpha}}{\Gamma(2k-1+\alpha)}+\dddd{ x^{2k-2}}{(2k-2)!} \rrrr),$$
$$y''(0)=\lim_{x\rightarrow 0}\dddd{d}{dx}F_{3}^{(1-\alpha)}(x)=-4\pi^2.$$
The exact solution of Equation III satisfies $y'(0)=0,y''(0)=-4\pi^2$. Let
$$\tilde{y}_1=y(0)+y'(0)h+y''(0)h^2/2.$$
From Taylor's formula $\tilde{y}_1$ is an approximation for $y(h)$ with accuracy $O\llll(h^3\rrrr)$.
The accuracy of numerical solution  $NS[45]$  with initial values $u_0=y_0,u_1=\tilde{y}_1$ is $O\llll( h^{3-\alpha}\rrrr)$. In Table 10 we compute the maximum error and the order of numerical solution $NS[45]$ of Equation I, Equation II  and Equation III. In Figure 3 we compare numerical solutions $NS[34],NS[40]$ and $NS[45]$ of Equation III and $\alpha=0.6$.
	\begin{figure}[t!]
  \centering
  \caption{Graph of the exact solution of Equation III  and  numerical solutions $NS[12]$-red,  and $NS[13]$-blue, and $NS[34]$-green for $h=0.1$ and $\alpha=0.6$.}
  \includegraphics[width=0.55\textwidth]{graph3.eps}
\end{figure}
\setlength{\tabcolsep}{0.5em}
{ \renewcommand{\arraystretch}{1.1}
\begin{table}[ht]
	\caption{Error and order of numerical solution $NS[13]$ of Equation I with $\alpha=0.25$, Equation II with $\alpha=0.5$ and Equation III with $\alpha=0.75$.}
	\small
	\centering
  \begin{tabular}{ l | c  c | c  c | c  c }
		\hline
		\hline
		\multirow{2}*{ $\quad \boldsymbol{h}$}  & \multicolumn{2}{c|}{{\bf Equation I}} & \multicolumn{2}{c|}{{\bf Equation II}}  & \multicolumn{2}{c}{{\bf Equation III}} \\
		\cline{2-7}
   & $Error$ & $Order$  & $Error$ & $Order$  & $Error$ & $Order$ \\
		\hline \hline
$0.003125$    & $7.9\times 10^{-8}$  & $2.7458$  & $7.6\times 10^{-8}$   & $2.4945$    & $0.00003552$         & $2.2495$       \\
$0.0015625$   & $1.2\times 10^{-8}$  & $2.7479$  & $1.3\times 10^{-8}$   & $2.4973$    & $7.5\times 10^{-6}$  & $2.2499$       \\
$0.00078125$  & $1.7\times 10^{-9}$  & $2.7490$  & $2.4\times 10^{-9}$   & $2.4987$    & $1.6\times 10^{-6}$  & $2.2500$        \\
$0.000390625$ & $2.6\times 10^{-10}$ & $2.7495$  & $4.2\times 10^{-10}$  & $2.4993$    & $3.3\times 10^{-7}$  & $2.2500$        \\
\hline
  \end{tabular}
	\end{table}
	}
\section{Conclusions}
In the present paper we obtained approximations of  the Caputo derivative of order $2-\alpha,2$ and $3-\alpha$ whose weights consist of terms which have power $-\alpha$ and $-1-\alpha$. In all experiments the accuracy of the numerical solutions  which use approximation \eqref{6_20} for the Caputo derivative is higher than the accuracy of the numerical solutions using approximations \eqref{3_10} and \eqref{5_30}. A question for future work is to construct an approximation of order $2-\alpha$ whose weights consist of terms which have power $-\alpha$, where the accuracy of the numerical solutions of Equation I, Equation II and Equation III is higher than the accuracy of the numerical solutions using the $L1$ approximation \eqref{3_10}.