Exact Augmented Lagrangian Duality for Mixed Integer Quadratic Programming

Abstract: Mixed integer quadratic programming (MIQP) is the problem of minimizing a convex quadratic function over mixed integer points in a rational polyhedron. This paper focuses on the augmented Lagrangian dual (ALD) for MIQP. ALD augments the usual Lagrangian dual with a weighted nonlinear penalty on the dualized constraints. We first prove that ALD will reach a zero duality gap asymptotically as the weight on the penalty goes to infinity under some mild conditions on the penalty function. We next show that a finite penalty weight is enough for a zero gap when we use any norm as the penalty function. Finally, we prove a polynomially bound on the weight on the penalty term to obtain a zero gap.

\section{An example appendix}

\clearpage
\bibliographystyle{siamplain}

\section{Introduction}
\label{sec:intro}
We consider the following rational (mixed) integer quadratic programming (MIQP) problem with decision variable $x\in\mathbb{R}^n$:
\begin{equation}\label{eq:mainproblem}
z^{\mathrm{IP}}:=\inf\{c^\top x+\frac{1}{2}x^\top Q x:Ax=b,x\in X\},
\end{equation}
where the parameters are: a rational symmetric positive semi-definite matrix $Q\in\mathbb{Q}^{n\times n}$, a rational matrix $A\in\mathbb{Q}^{m\times n}$, rational vectors $c\in \mathbb{Q}^n$ and $b\in \mathbb{Q}^m$, a mixed integer linear set $X$ such that
\[X=\{(x_1,x_2)\in\mathbb{R}^{n_1}\times \mathbb{Z}^{n_2}:Ex\leq f\},\]
where $E\in\mathbb{Q}^{m_2\times n}$ is a rational matrix and $f\in\mathbb{Q}^{m_2}$ is a rational vector with $n_1+n_2=n$.
We consider dualizing the constraints $Ax= b$.

While for continuous quadratic programming (QP), it is well known that even the classical Lagrangian dual (LD) will reach a zero duality gap and strong duality holds \cite{bertsekas2003convex}, it is not true for MIQP, as the integer variables introduce non-convexity. In fact, LD may have a non-zero duality gap for the problem.
Therefore, to close the gap, the idea of penalizing violation of the dualized constraints with a nonlinear penalty gives rise to the well known augmented Lagrangian dual (ALD), which is
\[z_\rho^{\mathrm{LD+}}:=\sup_\lambda\inf_{x\in X}\{c^\top x+\frac{1}{2}x^\top Q x+\lambda^\top(b-Ax)+\rho\psi(b-Ax)\},\]
where $\rho>0$ is the penalty weight, and $\psi(\cdot)$ is the penalty function which usually satisfies $\psi(0)=0$ and $\psi(u)>0$ if $u\neq 0$ \cite{rockafellar2009variational}.

Numerous papers have discussed ALD.
The paper \cite{rockafellar1974augmented} uses convex quadratic penalty functions for nonconvex programming, \cite{feizollahi2017exact} discusses the asymptotic zero duality gap and exact penalty representation for mixed integer \emph{linear} programming (MILP), \cite{burachik2017existence} discusses the optimality conditions for semi-infinite programming, and \cite{burachik2010duality} discusses exact penalization for general augmented Lagrangian.

It should be noted that an exact penalty representation usually requires a much restricted penalty function, like norm functions, see for example \cite{rockafellar2009variational}.
Norm function is used in \cite{burke1991exact} for exact penalization. The work \cite{huang2003unified} discusses exact penalty representation using level-bounded augmented functions and \cite{rubinov2002zero} considers the penalty function which is almost peak at zero. More recent works like \cite{feizollahi2017exact,burachik2017existence} apply sharp Lagrangian to different types of problems.

On the other hand, the size (for example, in binary coding) of the penalty weight is rarely discussed. While there are discussions for the size and computational complexity of MILP \cite{von1978bound,borosh1976bounds}, QP \cite{vavasis1990quadratic} and MIQP \cite{del2017mixed}, we might be able to utilize their ideas to show the small size of the penalty weight.

In this paper, we significantly generalize the results of \cite{feizollahi2017exact}. In particular, we
\begin{enumerate}
\item Prove that the duality gap of ALD will asymptotically reach zero under mild conditions as the penalty weight goes to infinity;
\item Prove that the duality gap will reach zero given that the penalty function is any norm, and the penalty weight is sufficiently large but still finite;
\item Prove that the size of the penalty weight which attains zero duality gap is polynomially bounded with respect to the problem data.
\end{enumerate}

The paper is organized as follows. In \Cref{sec:mainresults} we provide definitions and formal statement of main results of the paper. In \Cref{sec:preliminary} we present several key lemmas useful across the paper. In \Cref{sec:zerogap} we exhibit properties of ALD as the penalty weight goes to infinity, and show the (asymptotic) zero duality gap for a large class of penalty functions. In \Cref{sec:exactpenalty} we show a finite penalty weight whose size is polynomially bounded with respect to the input parameters, for which a zero duality gap is attained.
\section{Main Results}
\label{sec:mainresults}
In this section, we introduce some definitions and briefly present our main results

\begin{assumption}\label{ass:problem}
The MIQP \cref{eq:mainproblem} is feasible and the optimal value is bounded.
\end{assumption}

\begin{definition} The augmented Lagrangian relaxation is defined as
\[z_\rho^{\mathrm{LR+}}(\lambda):=\inf_{x\in X}\{c^\top x+\frac{1}{2}x^\top Q x+\lambda^\top(b-Ax)+\rho\psi(b-Ax)\},\]
where $\psi$ is a penalty function. The augmented Lagrangian dual is defined as
\[z_\rho^{\mathrm{LD+}}:=\sup_\lambda z_\rho^{\mathrm{LR+}}(\lambda)=\sup_\lambda\inf_{x\in X}\{c^\top x+\frac{1}{2}x^\top Q x+\lambda^\top(b-Ax)+\rho\psi(b-Ax)\}.\]
\end{definition}

\begin{definition} The continuous relaxation of \cref{eq:mainproblem} is denoted as $z^{\mathrm{NLP}}$
\[z^\mathrm{NLP}:=\inf\{c^\top x+\frac{1}{2}x^\top Q x: Ax=b,Ex\leq f,x\in\mathbb{R}^{n_1+n_2}\}.\]
\end{definition}

\begin{remark}
We use $\bar{\lambda}$ to denote the optimal dual variables (of $z^\mathrm{NLP}$) for the constraints $Ax=b$ and $\bar{\lambda}_E$ to denote the optimal dual variables for $Ex\leq f$. The existence of $\bar{\lambda}$ and $\bar{\lambda}_E$ is guaranteed by the boundedness of the continuous relaxation, which is given by \cref{lm:NLPQPIP}.
\end{remark}

\begin{remark}
For any $\rho$, $\lambda$, we have $z_\rho^{\mathrm{LR+}}(\lambda)\leq z_\rho^{\mathrm{LD+}}\leq z^\mathrm{IP}$. Moreover, we have $z^\mathrm{NLP}=\inf\{c^\top x+\frac{1}{2}x^\top Q x+\bar{\lambda}^\top(b-Ax):Ex\leq f,x\in\mathbb{R}^{n_1+n_2}\}\leq z_\rho^{\mathrm{LR+}}(\bar{\lambda}) \leq z_\rho^{\mathrm{LD+}}\leq z^\mathrm{IP}$.
\end{remark}

\begin{definition}
For a finite set of vectors $T=\{t_1,t_2,...,t_k\}$, $\mathrm{conv}(T)$, $\mathrm{cone}(T)$ and $\mathrm{int.cone}(T)$ are the convex hull, conical hull and integral conical hull of $T$, respectively. Here, $\mathrm{int.cone}(T):=\{\sum_{i=1}^k\mu_it_i:\mu_i\in\mathbb{Z}_+\}$.
\end{definition}

\begin{definition}
For any subset $T$ of a metric space, its diameter $\rm{\mathrm{diam}}$ is defined as ${\rm{\mathrm{diam}}}(T)=\sup _{a,b\in T}\|a-b\|$, where $\|\cdot\|$ is the metric associated with the space.
\end{definition}

\begin{definition} [\cite{del2017mixed}] Given an object $\mathcal{O}$ and another object $f(\mathcal{O})$ which is a function of it, we say that $f(\mathcal{O})$ has $\mathcal{O}$-small complexity, if the size (in standard binary encoding) of $f(\mathcal{O})$ is bounded above by a a polynomial function of the size of $\mathcal{O}$.
\end{definition}

\begin{definition} We use $\mathcal{F}$ to denote all input parameters of \cref{eq:mainproblem} including $E$, $f$, $c$, $Q$, $A$ and $b$. In addition, any object $q$ which is a function of $\mathcal{F}$ is said to have small complexity, if $q$ has $\mathcal{F}$-small complexity.
\end{definition}

Below we present the main theorems of the paper.

\begin{restatable}[Asymptotic Zero Duality Gap]{theorem}{AZDP}
\label{thm:main:zdg}
\label{thm:zdg}
Assume $\psi$ is proper, nonnegative, lower-semicontinuous and level-bounded, that is: $\psi(0)=0$; $\psi(u)>0$ for all $u\neq 0$; $\lim_{\delta\downarrow 0}{\rm{diam}}\{u:\psi(u)\leq \delta\}=0$; ${\rm{diam}}\{u:\psi(u)\leq \delta\}<\infty$ for all $\delta>0$. We have $\sup_{\rho>0}z^{\mathrm{LD+}}_\rho=z^{\mathrm{IP}}$.
\end{restatable}
\begin{restatable}[Exact Penalty Representation]{theorem}{EPR}
\label{thm:main:flexeald}
\label{thm:flexeald}
Suppose $\psi(\cdot)$ is any norm.
\begin{enumerate}[(a)]
\item There exists a $\rho^*$ of small complexity, such that $z^{\mathrm{LD+}}_{\rho^*}=z^{\mathrm{IP}}$.
\item Moreover, for all $\lambda$, there exists a finite $\rho^*(\lambda)$ of $\mathcal{F},\lambda$-small complexity, such that $z^\mathrm{LR+}_{\rho^*}(\lambda)=z^{\mathrm{IP}}$.
\end{enumerate}
\end{restatable}

We provide a flowchart that depicts how the preliminary results proved in \Cref{sec:preliminary} are put together to prove \cref{thm:main:zdg}.

\begin{tikzpicture}[node distance=2cm, text width=5.5cm]
\node (l12) [point] {\centering\cref{lm:NLPQPIP} \\ Equivalence of Boundedness of MIQP and its Continuous Relaxation};
\node (t17) [point, below of=l12, yshift = -0.3cm] {\centering\cref{prop:penaltylimit} \\ Approximation of the Penalty Term};
\node (t18) [point, below of=t17, yshift = 0.3cm] {\centering\cref{lm:penaltyrelax} \\ Equivalent Form of $z^\mathrm{LR+}_\rho(\bar{\lambda})$};
\node (l1314) [point, right of=l12, xshift = 4.5cm] {{\centering\cref{lm:Qdecomp}\\} Decomposition of Rational Mixed Integer Polyhedron \\ \centering\cref{lm:boundedref} \\ Bounded Region with Small Complexity};
\node (t20) [point, below of=l1314, yshift = -0.3cm] {\centering\cref{lm:zdg:boundary} \\ Adding a Uniform Bound on $x$ without Changing the Value};
\node (t19) [point, below of=t20, yshift = 0.3cm] {\centering\cref{thm:zdg}\\ Asymptotic Zero Duality Gap};

\draw [arrow](l12)--(t17);
\draw [arrow](t17)--(t18);
\draw [arrow](t18)--(t19);
\draw [arrow](l1314)--(t20);
\draw [arrow](t20)--(t19);
\end{tikzpicture}

Another flowchart is provided respect to the proof of \cref{thm:main:flexeald}.

\begin{tikzpicture}[node distance=2cm, text width=5.5cm]
\node (l12) [point] {\centering\cref{lm:NLPQPIP}\\ Equivalence of Boundedness of MIQP and its Continuous Relaxation};
\node (t21) [point, below of=l12, yshift = -0.3cm] {\centering\cref{thm:fixeald_ge}\\ A Sufficient Condition for Exact Penalty};
\node (t19) [point, below of=t21, yshift = 0.3cm] {\centering\cref{thm:zdg}\\ Asymptotic Zero Duality Gap};
\node (l1314) [point, right of=l12, xshift = 4.5cm] {{\centering\cref{lm:Qdecomp}\\} Decomposition of Rational Mixed Integer Polyhedron \\ \centering\cref{lm:boundedref} \\ Bounded Region with Small Complexity};
\node (t23) [point, below of=l1314, yshift = -0.3cm] {\centering\cref{lm:eald:boundary} \\ A Uniform Bound on $x$ Independent of $\rho$};
\node (t22) [point, below of=t23, yshift = 0.3cm] {\centering\cref{thm:fixeald}\\ Exact Penalty Representation for $L^\infty$ Norm};
\node (t24) [point, below of=t22] {\centering\cref{thm:flexeald}\\ Exact Penalty Representation for Any Norm};

\draw [arrow](l12)--(t21);
\draw [arrow](t19)--(t22);
\draw [arrow](l1314)--(t23);
\draw [arrow](t23)--(t22);
\draw [arrow](t22)--(t24);
\end{tikzpicture}
\section{Preliminary Results}
\label{sec:preliminary}
Several useful lemmas are presented in this section
\begin{lemma}[Equivalence of Boundedness of MIQP and its Continuous Relaxation]
\label{lm:NLPQPIP}
Suppose the MIQP is feasible (i.e. $z^{\mathrm{IP}}<+\infty$). Then the following three conditions are equivalent:
\begin{enumerate}
\item $z^{\mathrm{NLP}}$ is bounded.
\item $\inf\{c^\top x|Ax=0, Ex\leq 0, Qx=0\}$ is bounded.
\item $z^{\mathrm{IP}}$ is bounded.
\end{enumerate}
\end{lemma}

\begin{proof}
$1\Rightarrow3$ is obvious.

We first show $3\Rightarrow2$, or equivalently $\neg 2\Rightarrow \neg 3$. Note that the problem in 2 is always feasible. Assuming $\neg 2$, the problem $\{c^\top x\leq -1, Ax=0,Ex\leq 0,Qx=0\}$ is now feasible and there exists a rational solution since the problem is rational. Denote such a rational solution as $r$ and without loss of generality, we assume that $r$ is integral since we can scale $r$ with a positive coefficient.

Now select any feasible solution for 3, as $x$. Then we know that $x+tr$ is still feasible for 3 for any $t\in\mathbb{Z}_+$. In addition, $c^\top(x+tr)+\frac{1}{2}(x+tr)^\top Q (x+tr)=c^\top x+\frac{1}{2}x^\top Qx+tc^\top r\to-\infty$ as $t\to+\infty$. Therefore, we have 3 is unbounded, i.e. $3\Rightarrow 2$.

Next we show that $2\Rightarrow1$. Suppose that 2 holds. From Farkas Lemma, we know that $\exists \lambda_E\leq 0, \lambda_A, \lambda_Q$, such that
$\lambda_E^\top E+\lambda_A^\top A+\lambda_Q^\top Q=c^\top$. Now considering the NLP
\[
\begin{aligned}
z^{\mathrm{NLP}}=\inf\ &c^\top x+\frac{1}{2}x^\top Q x\\
\mathrm{s.t.}\ &Ax=b\\
&Ex\leq f\\
=\inf\ &(\lambda_E^\top E+\lambda_A^\top A+\lambda_Q^\top Q)x+\frac{1}{2}x^\top Qx\\
\mathrm{s.t.}\ &Ax=b\\
&Ex\leq f\\
\geq\inf\ &\lambda_E^\top f+\lambda_A^\top b + \lambda_Q^\top Qx+\frac{1}{2}x^\top Qx\\
\mathrm{s.t.}\ &Ax=b\\
&Ex\leq f.
\end{aligned}
\]
To show that $\lambda_Q^\top Q x +\frac{1}{2} x^\top Q x$ is bounded, we first write down the orthogonal decomposition of $Q$ as $R^\top\Lambda R$ where $R$ is orthogonal and $\Lambda$ is diagonal. Therefore, $\lambda_Q^\top Q x +\frac{1}{2} x^\top Q x=\sum_i \Lambda_{ii}(\lambda_Q^\top r_i+\frac{1}{2}x^\top r_i)x^\top r_i\geq \sum_i-\Lambda_{ii} (\lambda_Q^\top r_i)^2/2=-\lambda_Q^\top Q\lambda_Q/2$, where $r_i$ is the $i$-th row of the orthogonal matrix $R$. Note that the bound is attainable if we take $x=-\lambda_Q$. Therefore, we arrive at 1, i.e. $z^\mathrm{NLP}$ is bounded.
\end{proof}

\begin{remark} We note here that we are able to prove that the boundedness of the nonlinear integer problem implies boundedness of its continuous relaxation, using the fact that the data is rational. This is very similar to the Fundamental theorem of Integer Programming \cite{meyer1974existence}. Note that other similar results may be proven under different assumptions such as existence of integer point in the interior of  continuous relaxation, see \cite{dey2013some,moran2012strong}. Also see \cite{moran2018subadditive}.
\end{remark}

\begin{lemma}[Decomposition of Rational Mixed Integer Polyhedron]
\label{lm:Qdecomp}
Given a rational positive semidefinite matrix $Q$, any rational mixed integer polyhedron $P\cap(\mathbb{R}^{n_1}\times \mathbb{Z}^{n_2})=\{x:Cx\leq d\}\cap(\mathbb{R}^{n_1}\times \mathbb{Z}^{n_2})$ can be decomposed (with respect to $Q$) as $\cup_i (P_i\cap(\mathbb{R}^{n_1}\times \mathbb{Z}^{n_2})+\mathrm{int.cone}(R_i))$ satisfying the following properties:
\begin{enumerate}[(a)]
\item Each $P_i$ is a rational polytope.
\item Each $\mathrm{cone}(R_i)$ is a rational, simple and pointed cone.
\item For every $\mathrm{cone}(R_i)$, if a face $C'$ satisfies that $\exists x\in C'\backslash\{0\}$, $x^\top Qx=0$, then there exists an extreme ray $v$ of $C'$ with $v^\top Qv=0$.
\item Each polytope $P_i$ and each vector in $R_i$ has $P,Q$-small complexity.
\end{enumerate}
\end{lemma}

\begin{proof}
This lemma is a direct outcome of \cite[Proposition 1, Proposition 2, Lemma 2]{del2017mixed}.

First, if $P$ is not pointed, we can decompose $P$ into at most $2^{n_1+n_2}$ pointed rational mixed integer polyhedron by separating $x_k\leq 0$ and $x_k\geq 0$ for all $k$. Therefore, we simply assume $P$ is pointed henceforth.

Next, using \cite[Proposition 1]{del2017mixed}, we can decompose $P$ as $P=\cup_{i,K^1\in\mathcal{K}^1}(P^1_i+\mathrm{cone}(R^1_{K^1}))$, while conditions (a), (b), (d) are met.

Later, using \cite[Lemma 2]{del2017mixed}, we are able to decompose $\mathrm{cone}(R_k)$ into a union of rational, simple and pointed cones, which satisfies condition (c) and maintains (a), (b), (d). Therefore, $P=\cup_{i,K^2\in\mathcal{K}^2}(P^1_i+\mathrm{cone}(R^2_{K^2}))$.

Finally, we use \cite[Proposition 2]{del2017mixed} and decompose $(P^1_i+\mathrm{cone}(R^2_{K^2})) \cap(\mathbb{R}^{n_1}\times \mathbb{Z}^{n_2})$ into a mixed integer rational polytope plus an integer cone, which completes the proof.
\end{proof}

\begin{lemma}[Bounded Region with Small Complexity \cite{del2017mixed}]
\label{lm:boundedref}
Let $P \subseteq R^n$ be a polytope and $R\subseteq R^n$ be a finite set of vectors. Given a rational positive semidefinite matrix $Q$ of small complexity, suppose $P+\mathrm{cone}(R)$ satisfies the following properties:
\begin{enumerate}[(a)]
\item $P$ is a rational polytope.
\item $\mathrm{cone}(R)$ is a rational, simple and pointed cone.
\item $\forall x\in\mathrm{cone}(R)\backslash\{0\}$, $x^\top Qx>0$.
\item $P$ and each vector in $R$ has small complexity.
\end{enumerate}
Then, for any $\eta\in\mathbb{R}^n,\mu\in\mathbb{R}$ of small complexity, there exists $M$ of small complexity such that $\{x\in P+\mathrm{cone}(R):
\frac{1}{2}x^\top Qx+\eta^\top x\leq\mu\} \subset \{x:\|x\|\leq M\}$. In addition, such $M$ exists for any norm.
\end{lemma}

\begin{remark}
For any rational mixed integer polyhedron, \cref{lm:Qdecomp} provides a decomposition with respect to $Q$, while maintaining a small complexity. In addition, for any part of the decomposition, if no extreme ray $v$ has $v^\top Q v=0$ then no ray has $x^\top Q x=0$ (i.e. $x^\top Q x>0$ for any ray).

\cref{lm:boundedref} shows that under the conditions that $x^\top Q x>0$ for any ray, the optimal solution of the optimization problem $\{\min\frac{1}{2}x^\top Q x+\eta^\top x:x\in P+\mathrm{cone}(R)\}$ will have small complexity. In the lemma, this property is presented in the form of a feasibility problem.

The two lemmas will be needed for proving bounds in our proofs of theorems.
\end{remark}
\section{Asymptotic Zero Duality Gap}
\label{sec:zerogap}
In this section, we show that under mild conditions on the penalty function the ALD duality gap vanishes as the penalty weight $\rho$ goes to infinity.
\begin{assumption}[Conditions for Asymptotic Zero Duality Gap]
\label{assum:penalty}
We assume $\psi$ is proper, nonnegative, lower-semicontinuous and level-bounded, that is: $\psi(0)=0$; $\psi(u)>0$ for all $u\neq 0$; $\lim_{\delta\downarrow 0}{\rm{diam}}\{u:\psi(u)\leq \delta\}=0$; ${\rm{diam}}\{u:\psi(u)\leq \delta\}<\infty$ for all $\delta>0$.
\end{assumption}
\begin{proposition}[Approximation of the Penalty Term]
\label{prop:penaltylimit}
For given $\rho>0$ and $\epsilon>0$, define $w_{\rho,\epsilon}^*$ as
\begin{equation}
\label{eq:prop:penaltylimit}
\begin{aligned}
w_{\rho,\epsilon}^* :=\ \inf_{x,w}\ &w\\
\mathrm{s.t.}\ &x\in X,\\
\ &\psi(b-Ax)\leq w,\\
\ &c^\top x+\frac{1}{2}x^\top Qx+\bar{\lambda}^\top(b-Ax)+\rho w-z_\rho^{\mathrm{LR+}}(\bar{\lambda})\leq\epsilon.
\end{aligned}
\end{equation}
Then, the limit $w^*_\rho:=\lim_{\epsilon\downarrow 0}w_{\rho,\epsilon}^*$ exists and $\lim_{\rho\to+\infty}w^*_\rho=0$.
\end{proposition}
\begin{proof}First we need show that the problem \cref{eq:prop:penaltylimit} is well-defined, i.e is feasible and bounded. As a first step we show that $z_\rho^{\mathrm{LR+}}(\bar{\lambda})$ is finite. Observe that:
\[
\begin{aligned}
z_\rho^{\mathrm{LR+}}(\bar \lambda )\geq &\inf\{c^\top x+\frac{1}{2}x^\top Qx+\bar{\lambda}^\top (b-Ax):x\in X\}\\
\geq &\inf\{c^\top x+\frac{1}{2}x^\top Qx+\bar{\lambda}^\top (b-Ax):Ex\leq f\}\\
\geq &\inf\{c^\top x+\frac{1}{2}x^\top Qx+\bar{\lambda}^\top (b-Ax) +\bar{\lambda}^\top_E (f-Ex)\}\\
=&z^\mathrm{NLP},
\end{aligned}
\]
and the boundedness of $z^\mathrm{NLP}$ is given by \cref{lm:NLPQPIP}.

From the feasibility of the original problem we know that there exists an $x$ feasible for $z_\rho^{\mathrm{LR+}}(\bar{\lambda})$. Therefore, we are able to find $x\in X$ such that $c^\top x+\frac{1}{2}x^\top Qx+\bar{\lambda}^\top (b-Ax)+\psi(b-Ax)\leq \epsilon+ z_\rho^{\mathrm{LR+}}(\bar{\lambda})$, which means $(x,w=\psi(b-Ax))$ is feasible for \cref{eq:prop:penaltylimit}.
We also have $w^*_{\rho,\epsilon}\geq 0$ from the non-negativity of $\psi$. Thus, \cref{eq:prop:penaltylimit} is feasible and bounded.

In addition, we have $z_\rho^{\mathrm{LR+}}(\bar{\lambda})\leq z^{\mathrm{IP}}$ and for $x$ satisfying $Ex \leq f$ we have that $c^\top x+\frac{1}{2}x^\top Qx+\bar{\lambda}^\top(b-Ax)\geq \inf\{c^\top x+\frac{1}{2}x^\top Qx+\bar{\lambda}^\top (b-Ax) +\bar{\lambda}^\top_E (f-Ex)\}=z^\mathrm{NLP}$. Therefore,
\begin{align*}
w_{\rho,\epsilon}^*\leq& \frac{1}{\rho} \{z_\rho^{\mathrm{LR+}}(\bar{\lambda})+\epsilon -[c^\top x+\frac{1}{2}x^\top Qx+\bar{\lambda}^\top(b-Ax):x\in X]\}\\
\leq &\frac{1}{\rho}(z^{\mathrm{IP}}+\epsilon-z^{\mathrm{NLP}}).
\end{align*}

By taking $\epsilon\downarrow 0$ we have
\begin{equation}\label{eq:wrhostar}
0\leq w_\rho^*=\lim_{\epsilon\downarrow 0} w_{\rho,\epsilon}^*\leq\lim_{\epsilon\downarrow 0}\frac{1}{\rho}(z^{\mathrm{IP}}+\epsilon-z^{\mathrm{NLP}})=\frac{1}{\rho}(z^{\mathrm{IP}}-z^{\mathrm{NLP}}).\end{equation}
In addition, as $\epsilon\downarrow 0$ the feasible region of \cref{eq:prop:penaltylimit} becomes smaller, which indicates that $w_{\rho,\epsilon}^*$ is non-decreasing. Therefore $w_\rho^*=\lim_{\epsilon\downarrow 0} w_{\rho,\epsilon}^*$ exists.

By taking $\rho\to+\infty$ we therefore obtain $\lim_{\rho\to+\infty} w^*_\rho=0$.
\end{proof}

\begin{lemma}[Equivalent Form of $z^\mathrm{LR+}_\rho(\bar{\lambda})$]\label{lm:penaltyrelax}
Consider $w^*_\rho$ as in \cref{prop:penaltylimit} and define $\tilde{z}_\rho^{\mathrm{LR+}}(\bar{\lambda})$ as
\begin{equation}\label{eq:penaltyrelax:tildez}
\begin{aligned}
\tilde{z}_\rho^{\mathrm{LR+}}(\bar{\lambda}):=\ \inf_{x,w}\ &c^\top x +\frac{1}{2}x^\top Q x+\bar{\lambda}^\top (b-Ax)+\rho w\\
\mathrm{s.t.}\ &x\in X,\\
&\psi(b-Ax)\leq w,\\
&(1-\delta)w_\rho^*\leq w\leq(1+\delta)w_\rho^*.
\end{aligned}
\end{equation}
Then, for any $\delta\in(0,1)$,
\begin{equation}
\label{eq:LRptb}
\begin{aligned}
z_\rho^{\mathrm{LR+}}(\bar{\lambda})=&\tilde{z}_\rho^{\mathrm{LR+}}(\bar{\lambda})\\
\geq&\inf_x\ c^\top x+\frac{1}{2} x^\top Q x+\bar{\lambda}^\top(b-Ax)+\rho(1-\delta)w_\rho^*\\
&\mathrm{s.t.}\ x\in X,\\
&\ \ \ \ \ \psi(b-Ax)\leq (1+\delta)w_\rho^*,\\
\geq&\inf_x\ c^\top x+\frac{1}{2} x^\top Q x+\bar{\lambda}^\top(b-Ax)\\
&\mathrm{s.t.}\ x\in X,\\
&\ \ \ \ \ \psi(b-Ax)\leq (1+\delta)w_\rho^*.
\end{aligned}
\end{equation}
\end{lemma}
\begin{proof}
Note that the definition of $\tilde z_{\rho}^{\mathrm{LR+}} (\bar \lambda)$ is the same as that of $z_{\rho}^{\mathrm{LR+}} (\bar \lambda)$ except for the additional constraint $(1 - \delta)w^*_{\rho} \leq w \leq (1 + \delta)w^*_{\rho}$. Suppose $\alpha_\rho:=\tilde{z}_\rho^{\mathrm{LR+}}(\bar{\lambda})-z_\rho^{\mathrm{LR+}}(\bar{\lambda})>0$ by contradiction. Then, for all $(x,w)$ feasible to \cref{eq:penaltyrelax:tildez} we have
\[c^\top x +\frac{1}{2}x^\top Q x+\bar{\lambda}^\top (b-Ax)+\rho w\geq \tilde{z}_\rho^{\mathrm{LR+}}(\bar{\lambda})=z_\rho^{\mathrm{LR+}}(\bar{\lambda})+\alpha_\rho,\]
which implies $(x,w)$ is infeasible for \cref{eq:prop:penaltylimit} if $\epsilon<\alpha_\rho$. Hence, $w^*_{\rho,\epsilon}\notin ((1-\delta)w_\rho^*,(1+\delta)w_\rho^*$), a contradiction. Therefore $\tilde{z}_\rho^{\mathrm{LR+}}(\bar{\lambda})=z_\rho^{\mathrm{LR+}}(\bar{\lambda})$ and the inequalities \cref{eq:LRptb} are straightforward to verify.
\end{proof}

We are now ready to present the asymptotic zero duality gap.
\AZDP*
\begin{proof}
$z^{\mathrm{LD+}}_\rho$ does not decrease as $\rho$ increases. Therefore, it is then sufficient to show that $\sup_{\rho\geq 1}z^{\mathrm{LD+}}_\rho=z^{\mathrm{IP}}$ under the assumption.

Let $\delta\in(0,1)$, and we have
\begin{subequations}
\begin{align}
  z_\rho^{\mathrm{LD+}}&\geq z_\rho^{\mathrm{LR+}}(\bar{\lambda}) \nonumber\\
  &\geq\inf_{x}\{c^\top x+\frac{1}{2}x^\top Qx+\bar{\lambda}^\top(b-Ax):x\in X,\psi(b-Ax)\leq(1+\delta)w_\rho^*\} \label{eq:zdg:1a} \\
  &\geq\inf_{x}\{c^\top x+\frac{1}{2}x^\top Qx+\bar{\lambda}^\top(b-Ax):x\in X,\psi(b-Ax)\leq\frac{2}{\rho}(z^\mathrm{IP}-z^\mathrm{NLP})\} \label{eq:zdg:1b} \\
  &\geq\inf_{x}\{c^\top x+\frac{1}{2}x^\top Qx+\bar{\lambda}^\top(b-Ax):x\in X,\|b-Ax\|_\infty\leq\kappa_\rho\} (\geq z^\mathrm{NLP}) \label{eq:zdg:1c}

\end{align}
\end{subequations}
where $\kappa_\rho:=\mathrm{diam}\{u:\psi(u)\leq\frac{2}{\rho}(z^\mathrm{IP}-z^\mathrm{NLP})\}$ which is obviously non-increasing with respect to $\rho$. \cref{eq:zdg:1a} is guaranteed by \cref{lm:penaltyrelax}. \cref{eq:zdg:1b} is valid from \cref{eq:wrhostar} and \cref{eq:zdg:1c} comes from the level-boundedness of \cref{assum:penalty}.

We will need the following lemma that provides a uniform bound $M$ on \cref{eq:zdg:1c} for $x$ independent of $\rho$.

\begin{lemma}[Adding a Uniform Bound on $x$ without Changing the Value]\label{lm:zdg:boundary}
Under the assumption that $\rho\geq 1$, $\exists M>0$ independent of $\rho$, such that
\[
\begin{aligned}
&\inf_{x}\{c^\top x+\frac{1}{2}x^\top Qx+\bar{\lambda}^\top(b-Ax):x\in X,\|b-Ax\|_\infty\leq\kappa_\rho\}\\
=&\min_{x}\{c^\top x+\frac{1}{2}x^\top Qx+\bar{\lambda}^\top(b-Ax):x\in X,\|b-Ax\|_\infty\leq\kappa_\rho,\|x\|_\infty\leq M\}.
\end{aligned}
\]
\end{lemma}

A proof of \cref{lm:zdg:boundary} is provided later. From \cref{lm:zdg:boundary} we have $z_\rho^\mathrm{LD+}\geq\min_{x}\{c^\top x+\frac{1}{2}x^\top Qx+\bar{\lambda}^\top(b-Ax):x\in X,\|b-Ax\|_\infty\leq\kappa_\rho,\|x\|_\infty\leq M\}$. By taking $\rho\to+\infty$, we get
\begin{subequations}
\begin{align}
&\lim_{\rho\to+\infty}z_\rho^{\mathrm{LD+}} \nonumber\\
\geq&\lim_{\rho\to+\infty}\min_{x}\{c^\top x+\frac{1}{2}x^\top Qx+\bar{\lambda}^\top(b-Ax):x\in X,\|b-Ax\|_\infty\leq\kappa_\rho,\|x\|_\infty\leq M\} \nonumber\\
=&\lim_{\rho\to+\infty}\min_{\|x_2\|_\infty\leq M}\min_{\|x_1\|_\infty\leq M}\{c^\top x+\frac{1}{2}x^\top Qx+\bar{\lambda}^\top(b-Ax):x\in X,\|b-Ax\|_\infty\leq\kappa_\rho\} \nonumber\\
=&\min_{\|x_2\|_\infty\leq M}\lim_{\rho\to+\infty}\min_{\|x_1\|_\infty\leq M}\{c^\top x+\frac{1}{2}x^\top Qx+\bar{\lambda}^\top(b-Ax):x\in X,\|b-Ax\|_\infty\leq\kappa_\rho\} \label{eq:zdg:enumeration}\\
\geq&\min_{\|x_2\|_\infty\leq M}\min_{\|x_1\|_\infty\leq M}\{c^\top x+\frac{1}{2}x^\top Qx+\bar{\lambda}^\top(b-Ax):x\in X,\|b-Ax\|_\infty\leq\lim_{\rho\to+\infty}\kappa_\rho\} \label{eq:zdg:lsc} \\
\geq&\min_{\|x\|_\infty\leq M}\{c^\top x+\frac{1}{2}x^\top Qx+\bar{\lambda}^\top(b-Ax):x\in X,\|b-Ax\|_\infty=0\} \label{eq:zdg:level} \\
=&\min_{x}\{c^\top x+\frac{1}{2}x^\top Qx:x\in X,Ax=b,\|x\|_\infty\leq M\}\geq z^{\mathrm{IP}}, \nonumber
\end{align}
\end{subequations}
where \cref{eq:zdg:enumeration} follows from the finiteness of $x_2$ under $\|x_2\|_\infty\leq M$, \cref{eq:zdg:lsc} follows from the lower semi-continuity of the continuous quadratic programming \cite[Proposition 6.5.2]{bertsekas2003convex}
and \cref{eq:zdg:level} holds by \cref{assum:penalty}. Note that for any $\rho,\lambda$, $z_\rho^{\mathrm{LR+}}(\lambda)\leq z_\rho^{\mathrm{LD+}}\leq z^{\mathrm{IP}}$, and thus $\lim_{\rho\to+\infty}z_\rho^{\mathrm{LD+}}=z^\mathrm{IP}$.
\end{proof}

Note that by proving the theorem we also show that $\lim_{\rho\to+\infty}z_\rho^{\mathrm{LR+}}(\bar{\lambda})=z^\mathrm{IP}$ from the non-decreasing of $z_\rho^{\mathrm{LR+}}(\bar{\lambda})$ with respect to $\rho$.

We now complete the proof by proving \cref{lm:zdg:boundary}
\begin{proof}[Proof of \cref{lm:zdg:boundary}]
Note that $\|b-Ax\|_\infty\leq\kappa_\rho$ can be written as linear constraints. Hence, apply \cref{lm:Qdecomp} to the feasible region for $\rho=1$ of \cref{eq:zdg:1b} and we get a decomposition $\cup_i (P_i\cap(\mathbb{R}^{n_1}\times \mathbb{Z}^{n_2})+\mathrm{int.cone}(R_i))$ with the properties listed in the lemma. Note that for all $r\in R_i$ we have $Ar=0$ from the constraints $\|b-Ax\|_\infty\leq\kappa_1$. Therefore, the feasible region for any $\rho\geq 1$ can be written as
\[
\begin{aligned}
&(\cup_i (P_i\cap(\mathbb{R}^{n_1}\times \mathbb{Z}^{n_2})+\mathrm{int.cone}(R_i)))\cap\{x:\|b-Ax\|_\infty\leq\kappa_\rho\}\\
=&\cup_i ((P_i\cap(\mathbb{R}^{n_1}\times \mathbb{Z}^{n_2})+\mathrm{int.cone}(R_i))\cap\{x:\|b-Ax\|_\infty\leq\kappa_\rho\})\\
=&\cup_i\{y=x+\sum \mu_k r_k:x\in P_i\cap(\mathbb{R}^{n_1}\times \mathbb{Z}^{n_2}),r_k\in R_i, \mu_k\in \mathbb{Z}_+,\|b-Ay\|_\infty\leq \kappa_\rho\}\\
=&\cup_i\{y=x+\sum \mu_k r_k:x\in P_i\cap(\mathbb{R}^{n_1}\times \mathbb{Z}^{n_2}),r_k\in R_i, \mu_k\in \mathbb{Z}_+,\|b-Ax\|_\infty\leq \kappa_\rho\}\\
=&\cup_i (P_i\cap(\mathbb{R}^{n_1}\times \mathbb{Z}^{n_2})\cap\{x:\|b-Ax\|_\infty\leq\kappa_\rho\})+\mathrm{int.cone}(R_i)).
\end{aligned}
\]

Now consider the problem $\inf_x\{c^\top x+\frac{1}{2}x^\top Qx+\bar{\lambda}^\top(b-Ax):x\in P_i\cap(\mathbb{R}^{n_1}\times \mathbb{Z}^{n_2})\cap\{x:\|b-Ax\|_\infty\leq\kappa_\rho\})+\mathrm{int.cone}(R_i)\}$. If there exists $r\in R_i$ such that $r^\top Q r=0$ (i.e. $Qr=0$), then the feasible region can be rewritten as $P_i\cap(\mathbb{R}^{n_1}\times \mathbb{Z}^{n_2}\cap\{x:\|b-Ax\|_\infty\leq\kappa_\rho\})+\mathrm{int.cone}(R_i\backslash \{r\})+\{\mu r:\mu\in\mathbb{Z}_+\}$.

We can use $y+\mu r$ such that $y\in P_i\cap(\mathbb{R}^{n_1}\times \mathbb{Z}^{n_2}\cap\{x:\|b-Ax\|_\infty\leq\kappa_\rho\})+\mathrm{int.cone}(R_i\backslash \{r\})$ and $\mu\in\mathbb{Z}_+$ to represent $x$. The problem is therefore $\inf_{y,\mu}\{(c^\top-\bar{\lambda}^\top A) \mu+c^\top y+\frac{1}{2}y^\top Qy+\bar{\lambda}^\top(b-Ay):y\in P_i\cap(\mathbb{R}^{n_1}\times \mathbb{Z}^{n_2}\cap\{x:\|b-Ax\|_\infty\leq\kappa_\rho\})+\mathrm{int.cone}(R_i\backslash \{r\}), \mu\in\mathbb{Z}_+\}$. Optimize the problem over $\mu$ and we get $\mu=0$ an optimal solution (or the problem is unbounded, contrary to \cref{eq:zdg:1b}). Therefore, we can refine the feasible region by omitting all $r\in R_i$ such that $Qr=0$. Denote the set after the process as $R_i^J$. Note that this process is independent of the value of $\rho$, and hence we have
\[
\begin{aligned}
z^\mathrm{IP}\geq\inf_{x}\ &c^\top x+\frac{1}{2}x^\top Qx+\bar{\lambda}^\top(b-Ax)\ \mathrm{s.t.}\ x\in X,\|b-Ax\|_\infty\leq\kappa_\rho\\
=\inf_{x}\ &c^\top x+\frac{1}{2}x^\top Qx+\bar{\lambda}^\top(b-Ax)\\
\mathrm{s.t.}\ &x\in \cup_i (P_i\cap(\mathbb{R}^{n_1}\times \mathbb{Z}^{n_2})\cap\{x:\|b-Ax\|_\infty\leq\kappa_\rho\})+\mathrm{int.cone}(R_i^J)).
\end{aligned}
\]

In addition, from (c) of \cref{lm:Qdecomp}, for all $x\in \mathrm{cone}(R_i^J)\backslash\{0\}$, we have $x^\top Q x>0$. Let
\[
V_i=\{x\in (P_i+\mathrm{cone}(R_i^J)):c^\top x+\frac{1}{2}x^\top Qx+\bar{\lambda}^\top(b-Ax)-(z^\mathrm{IP}+1)\leq 0\}.
\]
Note that the definition on $V_i$ is independent of $\rho$. Note that $(P_i\cap(\mathbb{R}^{n_1}\times \mathbb{Z}^{n_2})\cap\{x:\|b-Ax\|_\infty\leq\kappa_\rho\})+\mathrm{int.cone}(R_i^J))\subseteq (P_i+\mathrm{cone}(R_i^J))$ and we have
\[
\begin{aligned}
z^\mathrm{IP}\geq \inf_{x}\ &c^\top x+\frac{1}{2}x^\top Qx+\bar{\lambda}^\top(b-Ax)\\
\mathrm{s.t.}\ &x\in \cup_i (P_i\cap(\mathbb{R}^{n_1}\times \mathbb{Z}^{n_2})\cap\{x:\|b-Ax\|_\infty\leq\kappa_\rho\})+\mathrm{int.cone}(R_i^J))\\
=\inf_{x}\ &c^\top x+\frac{1}{2}x^\top Qx+\bar{\lambda}^\top(b-Ax)\\
\mathrm{s.t.}\ &x\in \cup_i (P_i\cap(\mathbb{R}^{n_1}\times \mathbb{Z}^{n_2})\cap\{x:\|b-Ax\|_\infty\leq\kappa_\rho\})+\mathrm{int.cone}(R_i^J))\\
&c^\top x+\frac{1}{2}x^\top Qx+\bar{\lambda}^\top(b-Ax)\leq z^\mathrm{IP}+1\\
\geq \inf_{x}\ &c^\top x+\frac{1}{2}x^\top Qx+\bar{\lambda}^\top(b-Ax)\\
\mathrm{s.t.}\ &x\in \cup_i (P_i\cap(\mathbb{R}^{n_1}\times \mathbb{Z}^{n_2})\cap\{x:\|b-Ax\|_\infty\leq\kappa_\rho\})+\mathrm{int.cone}(R_i^J))\\
&x\in \cup_i V_i.\\
\end{aligned}
\]

Using \cref{lm:boundedref} we have that there exists $M_i>0$ such that $V_i\in\{x:\|x\|_\infty\leq M_i\}$. Take $M=\max \{M_i\}$, which is independent of $\rho$, and we have
\[
\begin{aligned}
\inf_{x}\ &c^\top x+\frac{1}{2}x^\top Qx+\bar{\lambda}^\top(b-Ax)\\
\mathrm{s.t.}\ &x\in \cup_i (P_i\cap(\mathbb{R}^{n_1}\times \mathbb{Z}^{n_2})\cap\{x:\|b-Ax\|_\infty\leq\kappa_\rho\})+\mathrm{int.cone}(R_i^J))\\
&x\in \cup_i V_i\\
\geq \inf_{x}\ &c^\top x+\frac{1}{2}x^\top Qx+\bar{\lambda}^\top(b-Ax)\\
\mathrm{s.t.}\ &x\in \cup_i (P_i\cap(\mathbb{R}^{n_1}\times \mathbb{Z}^{n_2})\cap\{x:\|b-Ax\|_\infty\leq\kappa_\rho\})+\mathrm{int.cone}(R_i^J))\\
&\|x\|_\infty\leq M\\
\geq\inf_{x}\ &c^\top x+\frac{1}{2}x^\top Qx+\bar{\lambda}^\top(b-Ax)\ \mathrm{s.t.}\ x\in X, \|b-Ax\|_\infty\leq \kappa_\rho,\|x\|_\infty\leq M.
\end{aligned}
\]

While $\inf_{x}\{c^\top x+\frac{1}{2}x^\top Qx+\bar{\lambda}^\top(b-Ax)\ \mathrm{s.t.}\ x\in X, \|b-Ax\|_\infty\leq \kappa_\rho,\|x\|_\infty\leq M\}\geq z^\mathrm{NLP}$ is bounded and the values of $x_2$ here is finite, we can therefore replace $\inf$ by $\min$.

Therefore,
\[
\begin{aligned}
&\inf_{x}\{c^\top x+\frac{1}{2}x^\top Qx+\bar{\lambda}^\top(b-Ax):x\in X,\|b-Ax\|_\infty\leq\kappa_\rho\}\\
\geq&\min_{x}\{c^\top x+\frac{1}{2}x^\top Qx+\bar{\lambda}^\top(b-Ax):x\in X,\|b-Ax\|_\infty\leq\kappa_\rho,\|x\|_\infty\leq M\}.
\end{aligned}
\]

Since it is obvious that
\[
\begin{aligned}
&\inf_{x}\{c^\top x+\frac{1}{2}x^\top Qx+\bar{\lambda}^\top(b-Ax):x\in X,\|b-Ax\|_\infty\leq\kappa_\rho\}\\
\leq&\min_{x}\{c^\top x+\frac{1}{2}x^\top Qx+\bar{\lambda}^\top(b-Ax):x\in X,\|b-Ax\|_\infty\leq\kappa_\rho,\|x\|_\infty\leq M\},
\end{aligned}
\]
thus equality holds and the proof is completed.
\end{proof}
\section{Exact Penalty Representation}
\label{sec:exactpenalty}
In this section, we will discuss conditions for an exact penalty representation. To begin with, a sufficient condition is given. We later prove the sufficiency of using norm as the penalty function for an exact penalty, while noting that a norm function always satisfies \cref{assum:penalty}.

\begin{theorem}[A Sufficient Condition for Exact Penalty]\label{thm:fixeald_ge}
Under \cref{ass:problem} (MIQP is feasible and the optimal value is bounded), if there exists $\delta$, such that
\[\inf\{\psi(b-Ax):x\in X, Ax\neq b\}\geq\delta>0\]
and $\psi(0)=0$, then there exists a finite $\rho^*$ such that $z^{\mathrm{LR+}}_{\rho^*}(\bar{\lambda})=z^\mathrm{IP}$, which also gives $z^\mathrm{LD+}_{\rho^*}=z^\mathrm{IP}$.
\end{theorem}
\begin{proof}
Under \cref{ass:problem}, using \cref{lm:NLPQPIP}, we have $z^\mathrm{NLP}$ is bounded. Thus, choose a feasible point $\tilde{x}$ for the MIQP and set $$\rho^*=\frac{1}{\delta}(c^\top \tilde{x}+\frac{1}{2}\tilde{x}^\top Q \tilde{x}-z^\mathrm{NLP})<\infty,$$
we next show that $\rho^*$ satisfies our requirements.

First of all, as $z^\mathrm{NLP}$ bounded and $c^\top \tilde{x}+\frac{1}{2}\tilde{x}^\top Q \tilde{x}\geq z^\mathrm{IP}$, we have $\rho^*\in[0,+\infty)$. Clearly $z^{\mathrm{LR+}}_{\rho^*}(\bar{\lambda})\leq z^\mathrm{IP}$. We next show that $z^{\mathrm{LR+}}_{\rho^*}(\bar{\lambda})\geq z^\mathrm{IP}$.

For $x\in X$ with $Ax=b$, we have $$c^\top x+\frac{1}{2}x^\top Qx+\bar{\lambda}^\top(b-Ax)+\rho^*\psi(b-Ax)=c^\top x+\frac{1}{2}x^\top Qx\geq z^\mathrm{IP}.$$

For $x\in X$ with $Ax\neq b$, we have $$c^\top x+\frac{1}{2}x^\top Qx+\bar{\lambda}^\top(b-Ax)\geq z^\mathrm{NLP}$$
from the strong duality results for QP. Thus,
$$c^\top x+\frac{1}{2}x^\top Qx+\bar{\lambda}^\top(b-Ax)+\rho^*\psi(b-Ax)\geq z^\mathrm{NLP}+\rho^*\delta = c^\top \tilde{x}+\frac{1}{2}\tilde{x}^\top Q \tilde{x}\geq z^\mathrm{IP}.$$

Therefore, we have for $x\in X$, $c^\top x+\frac{1}{2}x^\top Qx+\bar{\lambda}^\top(b-Ax)+\rho^*\psi(b-Ax)\geq z^\mathrm{IP}$ and thus $z^{\mathrm{LR+}}_{\rho^*}(\bar{\lambda})=z^\mathrm{IP}$.
\end{proof}

We now present the exact penalty results for $\psi(\cdot)=\|\cdot\|_\infty$.
\begin{theorem}[Exact Penalty Representation for $L^\infty$-Norm]\label{thm:fixeald}
Assuming $\psi(\cdot)=\|\cdot\|_\infty$, there exists a finite $\rho^*(\bar{\lambda})$ of small complexity, such that $z_{\rho^*}^{\mathrm{LR+}}(\bar{\lambda})=z^{\mathrm{IP}}$.
\end{theorem}
\begin{proof}
It is sufficient to find a finite $\rho^*(\bar{\lambda})$ polynomially bounded, such that $z_{\rho^*}^{\mathrm{LR+}}(\bar{\lambda})\geq z^{\mathrm{IP}}$. Since $z^{\mathrm{LR+}}_\rho(\bar{\lambda})$ is non-decreasing with $\rho$ increasing, without loss of generality, we only consider $\rho\geq 1$. In addition, from \cite[Theorem 4]{del2017mixed}, $z^\mathrm{IP}$, $z^\mathrm{NLP}$ and $\bar{\lambda}$ have $\mathcal{F}$-small complexity.

The constraints $\|b-Ax\|_\infty\leq w$ can be written as $-\mathbf{1}w\leq b-Ax\leq \mathbf{1}w$. Therefore,
\begin{equation}\label{eq:LRp:origin}
\begin{aligned}
z_\rho^{\mathrm{LR+}}(\bar{\lambda})=\inf_{x,w}\ &(c^\top -\bar{\lambda}^\top A)x+\frac{1}{2}x^\top Qx+\rho w+\bar{\lambda}^\top b\\
\mathrm{s.t.}\ &Ax-\mathbf{1} w\leq b,\\
&-Ax-\mathbf{1} w\leq b,\\
&Ex\leq f,\\
&x\in\mathbb{R}^{n_1}\times \mathbb{Z}^{n_2}.
\end{aligned}
\end{equation}

The following lemma shows a uniform bound $M$ of small complexity can be put on $x$ independent of $\rho$.
\begin{lemma}[A Uniform Bound on $x$ Independent of $\rho$]\label{lm:eald:boundary}
Under the assumption that $\rho\geq 1$, there exists $M>0$ independent of $\rho$ and of small complexity, such that
\begin{equation}\label{eq:LRp:bounded}
\begin{aligned}
z_\rho^{\mathrm{LR+}}(\bar{\lambda})=\inf_{x,w}\ &(c^\top -\bar{\lambda}^\top A)x+\frac{1}{2}x^\top Qx+\rho w+\bar{\lambda}^\top b\\
\mathrm{s.t.}\ &Ax-\mathbf{1} w\leq b,\\
&-Ax-\mathbf{1} w\leq b,\\
&Ex\leq f,\\
&\|x\|_\infty\leq M,\\
&x\in\mathbb{R}^{n_1}\times \mathbb{Z}^{n_2}.
\end{aligned}
\end{equation}
\end{lemma}
A proof of \cref{lm:eald:boundary} is provided later. We next rewrite $x=(x_1,x_2)$ and separate $A,E,c$ respectively. We also rewrite
\[Q=\begin{bmatrix}Q^{(11)}&Q^{(12)}\\mathbb{Q}^{(21)}&Q^{(22)}\end{bmatrix}.\]
Note that $Q^{(11)}$ is also positive semi-definite. Therefore, the problem can be rewrite as
\begin{equation}\label{eq:LRp:separatebounded}
\begin{aligned}
z_\rho^{\mathrm{LR+}}(\bar{\lambda})=\inf_{x_1,x_2,w}\ &(c_1^\top -\bar{\lambda}^\top A_1+x_2^\top Q^{(21)})x_1+\frac{1}{2}x_1^\top Q^{(11)}x_1\\
&+\rho w+\bar{\lambda}^\top b+(c_2^\top-\bar{\lambda}^\top A_2)x_2+\frac{1}{2}x_2^\top Q^{(22)}x_2\\
\mathrm{s.t.}\ &A_1x_1-\mathbf{1} w\leq -A_2x_2 + b,\\
&-A_1x_1-\mathbf{1} w\leq A_2x_2 -b,\\
&E_1x_1\leq f-E_2x_2,\\
&x_1\leq\mathbf{1}M,-x_1\leq \mathbf{1}M,\\
&\|x_2\|_\infty\leq M,\\
&x_1\in\mathbb{R}^{n_1},x_2\in\mathbb{Z}^{n_2}\\
\end{aligned}
\end{equation}
Denote $V=\{v\in\mathbb{Z}^{n_2}: \|v\|_\infty\leq M\}$. In addition, we use $z_\rho^{\mathrm{LR+}}(\bar{\lambda},x_2)$ to denote $z_\rho^{\mathrm{LR+}}(\bar{\lambda})$ while fixing $x_2$. Therefore, $z_\rho^{\mathrm{LR+}}(\bar{\lambda})=\min_{x_2\in V}z_\rho^{\mathrm{LR+}}(\bar{\lambda},x_2)$. Note that $z_\rho^{\mathrm{LR+}}(\bar{\lambda},x_2)$ is still non-decreasing with respect to $\rho$. Therefore, from \cref{thm:zdg} we have $z^\mathrm{IP}=\lim_{\rho\to+\infty}\min_{x_2\in V}z_\rho^{\mathrm{LR+}}(\bar{\lambda},x_2)=\min_{x_2\in V}\lim_{\rho\to+\infty}z_\rho^{\mathrm{LR+}}(\bar{\lambda},x_2)$, or equivalently we have that $\lim_{\rho\to+\infty}z_\rho^{\mathrm{LR+}}(\bar{\lambda},x_2)\geq z^\mathrm{IP}$.

For arbitrary $x_2\in V$, the dual problem of \cref{eq:LRp:separatebounded} (with respect to $x_1,w$) is therefore
\[
\begin{aligned}
z^{\text{DRD+}}_\rho(\bar{\lambda},x_2):&=\sup_{y_1,y_2,y_3,y_4,y_5\geq 0}\inf_{x_1,w}(c_1^\top -\bar{\lambda}^\top A_1+x_2^\top Q^{(21)})x_1+\frac{1}{2}x_1^\top Q^{(11)}x_1\\
&+\rho w+\bar{\lambda}^\top b+(c_2^\top-\bar{\lambda}^\top A_2)x_2+\frac{1}{2}x_2^\top Q^{(22)}x_2\\
&+y_1^\top(A_1x_1-\mathbf{1} w+A_2x_2-b)-y_2^\top(A_1x_1+\mathbf{1} w+A_2x_2-b)\\
&+y_3^\top(E_1x_1-f+E_2x_2)+y_4^\top(x_1-\mathbf{1}M)+y_5^\top(-x_1-\mathbf{1}M).
\end{aligned}
\]
Note that the problem $\inf_{x_1,w}$ is bounded if and only if $(y_1+y_2)^\top \mathbf{1}=\rho$ and there exists $ \nu$, s.t. $c_1^\top -\bar{\lambda}^\top A_1+x_2^\top Q^{(21)}+(y_1-y_2)^\top A_1+y_3^\top E_1+(y_4-y_5)^\top=\nu^\top Q^{(11)}$. Therefore the problem is
\[
\begin{aligned}
z^{\text{DRD+}}_\rho(\bar{\lambda},x_2)=\sup_{y,\nu}\ &- \frac{1}{2}\nu^\top Q^{(11)}\nu+(A_2x_2-b)^\top y_1-(A_2x_2-b)^\top y_2\\
&+(E_2x_2-f)^\top y_3-M\mathbf{1}^\top(y_4+y_5)\\
&+\bar{\lambda}^\top b+(c_2^\top-\bar{\lambda}^\top A_2)x_2+\frac{1}{2}x_2^\top Q^{(22)}x_2\\
\mathrm{s.t.}\ &y_1,y_2,y_3,y_4,y_5\geq 0,\\
& \mathbf{1}^\top (y_1+y_2)=\rho,\\
& c- A_1^\top \bar{\lambda}+Q^{(12)} x_2+A_1^\top (y_1-y_2)+E_1^\top y_3 +y_4-y_5=Q^{(11)\top} \nu.
\end{aligned}
\]

By strong duality we have $z_\rho^{\mathrm{LR+}}(\bar{\lambda},x_2)=z^{\text{DRD+}}_\rho(\bar{\lambda},x_2)$. Therefore, we have $\lim_{\rho\to+\infty}z_\rho^{\text{DRD+}}(\bar{\lambda},x_2)=\lim_{\rho\to+\infty}z_\rho^{\mathrm{LR+}}(\bar{\lambda},x_2)\geq z^\mathrm{IP}$. Now consider the following problem with respect to $(\xi,y,\nu,\rho)$:
\[
\begin{aligned}
\min\ &\xi\\
\mathrm{s.t.}\ &- \frac{1}{2}\nu^\top Q^{(11)}\nu+(A_2x_2-b)^\top y_1-(A_2x_2-b)^\top y_2\\
&+(E_2x_2-f)^\top y_3-M\mathbf{1}^\top(y_4+y_5)\\
&+\bar{\lambda}^\top b+(c_2^\top-\bar{\lambda}^\top A_2)x_2+\frac{1}{2}x_2^\top Q^{(22)}x_2+\xi\geq z^\mathrm{IP},\\
&y_1,y_2,y_3,y_4,y_5,\xi\geq 0,\ \rho\geq 1,\\
& \mathbf{1}^\top (y_1+y_2)=\rho,\\
& c- A_1^\top \bar{\lambda}+Q^{(12)} x_2+A_1^\top (y_1-y_2)+E_1^\top y_3 +y_4-y_5=Q^{(11)\top} \nu.
\end{aligned}
\]
The above problem is a quadratically constrained quadratic programming (QCQP) with convex constraints and affine objective function. The existence of an optimal solution is guaranteed by the finiteness and feasibility of the problem \cite{bertsekas2003convex}. In addition, as $\lim_{\rho\to+\infty}z_\rho^{\text{DRD+}}(\bar{\lambda},x_2)\geq z^\mathrm{IP}$ there exists a sequence of $(\xi^k,y^k,\nu^k,\rho^k)$ feasible to the problem such that $\xi^k\to 0$. Therefore, the optimal value is $0$ and an optimal solution $(0,y^*,\nu^*,\rho^*)$ exists, which guarantees the feasibility of the following problem with respect to $(y,\nu,\rho)$:
\[
\begin{aligned}
\rho^*(x_2):=\min\ &\rho\\
\mathrm{s.t.}\ &- \frac{1}{2}\nu^\top Q^{(11)}\nu+(A_2x_2-b)^\top y_1-(A_2x_2-b)^\top y_2\\
&+(E_2x_2-f)^\top y_3-M\mathbf{1}^\top(y_4+y_5)\\
&+\bar{\lambda}^\top b+(c_2^\top-\bar{\lambda}^\top A_2)x_2+\frac{1}{2}x_2^\top Q^{(22)}x_2\geq z^\mathrm{IP},\\
&y_1,y_2,y_3,y_4,y_5\geq 0,\ \rho\geq 1,\\
& \mathbf{1}^\top (y_1+y_2)=\rho,\\
& c- A_1^\top \bar{\lambda}+Q^{(12)} x_2+A_1^\top (y_1-y_2)+E_1^\top y_3 +y_4-y_5=Q^{(11)\top} \nu.
\end{aligned}
\]
Similarly, the finiteness and feasibility of the problem guarantees the existence of the optimal solution. Therefore, $\rho^*(x_2)$ is well defined and from \cite[Theorem 4]{del2017mixed}, $\rho^*(x_2)$ has small complexity. In addition, we have $z^{\text{DRD+}}_{\rho^*(x_2)}(\bar{\lambda},x_2)\geq z^\mathrm{IP}$.

Now let $\rho^*=\max_{x_2\in V}\rho^*(x_2)$ of small complexity, we have $z^{\text{DRD+}}_{\rho^*} (\bar{\lambda},x_2)\geq z^{\text{DRD+}}_{\rho^*(x_2)}(\bar{\lambda},x_2)\geq z^\mathrm{IP}$ for all $x_2\in V$. Hence, $z^{\mathrm{LR+}}_{\rho^*} (\bar{\lambda})=\min_{x_2\in V}z^{\mathrm{LR+}}_{\rho^*} (\bar{\lambda},x_2)=\min_{x_2\in V}z^{\text{DRD+}}_{\rho^*} (\bar{\lambda},x_2)\geq z^\mathrm{IP}$.

Note that for any $\rho,\lambda$, $z_\rho^{\mathrm{LR+}}(\lambda)\leq z_\rho^{\mathrm{LD+}}\leq z^{\mathrm{IP}}$, and thus $z_{\rho^*}^\mathrm{LR+}(\bar{\lambda})=z_{\rho^*}^{\mathrm{LD+}}=z^\mathrm{IP}$.
\end{proof}

We next complete the proof by proving \cref{lm:eald:boundary}.
\begin{proof}[Proof of \cref{lm:eald:boundary}]
Consider for $\rho\geq 1$, $w_M:=2(z^\mathrm{IP}-z^\mathrm{NLP}) \geq 2w^*_\rho$, where the inequality follows \cref{eq:wrhostar}, and define
\[
\begin{aligned}
\hat{z}_\rho^{\mathrm{LR+}}(\bar{\lambda})=\inf_{x,w}\ &(c^\top -\bar{\lambda}^\top A)x+\frac{1}{2}x^\top Qx+\rho w+\bar{\lambda}^\top b\\
\mathrm{s.t.}\ &Ax-\mathbf{1} w\leq b,\\
&-Ax-\mathbf{1} w\leq -b,\\
&Ex\leq f,\\
&w\leq w_M,\\
&x\in\mathbb{R}^{n_1}\times \mathbb{Z}^{n_2}.
\end{aligned}\]
Denote the feasible region for this problem as $P$ and the feasible region for the original problem \cref{eq:LRp:origin} as $P^o$. Clearly, $P\subseteq P^o$ and thus $\hat{z}_\rho^{\mathrm{LR+}}(\bar{\lambda})\geq z_\rho^{\mathrm{LR+}}(\bar{\lambda})$. Similarly as $P$ is larger than the feasible region of \cref{eq:penaltyrelax:tildez}, we have $\hat{z}_\rho^{\mathrm{LR+}}(\bar{\lambda})\leq \tilde{z}_\rho^{\mathrm{LR+}}(\bar{\lambda})=z_\rho^{\mathrm{LR+}}(\bar{\lambda})$ and thus $\hat{z}_\rho^{\mathrm{LR+}}(\bar{\lambda}) = z_\rho^{\mathrm{LR+}}(\bar{\lambda})$.

Now apply \cref{lm:Qdecomp} to $P$ and we get a decomposition $P=\cup_i (P_i\cap(\mathbb{R}^{n_1}\times \mathbb{Z}^{n_2}\times \mathbb{R})+\mathrm{int.cone}(R_i))$ with the properties listed in the lemma. Note that the decomposition applies to all $\rho$.

Note that the problem is bounded from the boundedness of $z_\rho^{\mathrm{LR+}}$ and for all $r\in R_i$ the $w$-component is $0$ (from the constraints $w\leq w_M$, so we can omit the $w$-component for any vector in $R_i$, or simply denote it as $R_i\times \{0\}$.

Similar to the proof to \cref{lm:zdg:boundary}, when we solve the problem $\inf_{x,w}\{(c^\top -\bar{\lambda}^\top A)x+\frac{1}{2}x^\top Qx+\rho w+\bar{\lambda}^\top b:(x,w)\in P_i\cap(\mathbb{R}^{n_1}\times \mathbb{Z}^{n_2}\times \mathbb{R})+\mathrm{int.cone}(R_i)\times \{0\}\}$, if there exists $r\in R_i$ such that $r^\top Q r=0$ (i.e. $Qr=0$), the feasible region can be decomposed as as $P_i\cap(\mathbb{R}^{n_1}\times \mathbb{Z}^{n_2}\times \mathbb{R})+\mathrm{int.cone}(R_i\backslash \{r\})\times\{0\}+\{\mu r:\mu\in\mathbb{Z}_+\}\times \{0\}$. Optimize the problem over $\mu$ and we get $\mu=0$ an optimal solution (otherwise the problem will be unbounded). Therefore, we can refine the feasible region by omitting all $r\in R_i$ such that $Qr=0$. Denote the set after the process as $R_i^J$. Note that this process is independent of $\rho$, and hence we have
\[
\begin{aligned}
z_\rho^{\mathrm{LR+}}(\bar{\lambda})=\inf_{x,w}\ &(c^\top -\bar{\lambda}^\top A)x+\frac{1}{2}x^\top Qx+\rho w+\bar{\lambda}^\top b\\
\mathrm{s.t.}\ &(x,w)\in \cup_i(P_i\cap(\mathbb{R}^{n_1}\times \mathbb{Z}^{n_2}\times \mathbb{R})+\mathrm{int.cone}(R_i^J)\times\{0\}).
\end{aligned}\]
Now, from (c) of \cref{lm:Qdecomp}, for all $x\in \mathrm{cone}(R_i^J)\backslash\{0\}$, we have $x^\top Q x>0$
Let
\[
\begin{aligned}
V_i=\{&(x,w)\in (P_i+\mathrm{cone}(R^J_i)\times\{0\}):\\
&(c^\top -\bar{\lambda}^\top A)x+\frac{1}{2}x^\top Qx+w+\bar{\lambda}^\top b\leq z^\mathrm{IP}+1\}.
\end{aligned}
\]
Note that the definition on $V_i$ is independent of $\rho$. Therefore, as $\rho\geq 1$ we have
\[\begin{aligned}
z_\rho^{\mathrm{LR+}}(\bar{\lambda})=\inf_{x,w}\ &(c^\top -\bar{\lambda}^\top A)x+\frac{1}{2}x^\top Qx+\rho w+\bar{\lambda}^\top b\\
\mathrm{s.t.}\ &(x,w)\in \cup_i(P_i\cap(\mathbb{R}^{n_1}\times \mathbb{Z}^{n_2}\times \mathbb{R})+\mathrm{int.cone}(R_i^J)\times\{0\})\\
=\inf_{x,w}\ &(c^\top -\bar{\lambda}^\top A)x+\frac{1}{2}x^\top Qx+\rho w+\bar{\lambda}^\top b\\
\mathrm{s.t.}\ &(x,w)\in \cup_i(P_i\cap(\mathbb{R}^{n_1}\times \mathbb{Z}^{n_2}\times \mathbb{R})+\mathrm{int.cone}(R_i^J)\times\{0\})\\
&(c^\top -\bar{\lambda}^\top A)x+\frac{1}{2}x^\top Qx+w+\bar{\lambda}^\top b\leq z^\mathrm{IP}+1\\
\geq \inf_{x,w}\ &(c^\top -\bar{\lambda}^\top A)x+\frac{1}{2}x^\top Qx+\rho w+\bar{\lambda}^\top b\\
\mathrm{s.t.}\ &(x,w)\in \cup_i(P_i\cap(\mathbb{R}^{n_1}\times \mathbb{Z}^{n_2}\times \mathbb{R})+\mathrm{int.cone}(R_i^J)\times\{0\})\\
&(x,w)\in\cup_i V_i.
\end{aligned}\]

From \cref{lm:boundedref}, there exists $ M_i>0$, such that $V_i\subseteq \{(x,w):\|(x,w)\|_\infty\leq M_i\}$ and  $M_i$ has small complexity. Therefore, $V_i \subset \{(x,w):\|x\|_\infty\leq M_i\}$. Let $M=\max\{M_i\}$ (which is again independent of $\rho$ and has small complexity) and we have
\[
\begin{aligned}
z_\rho^{\mathrm{LR+}}(\bar{\lambda})\geq \inf_{x,w}\ &(c^\top -\bar{\lambda}^\top A)x+\frac{1}{2}x^\top Qx+\rho w+\bar{\lambda}^\top b\\
\mathrm{s.t.}\ &(x,w)\in \cup_i(P_i\cap(\mathbb{R}^{n_1}\times \mathbb{Z}^{n_2}\times \mathbb{R})+\mathrm{int.cone}(R_i^J)\times\{0\})\\
&(x,w)\in\cup_i V_i\\
\geq \inf_{x,w}\ &(c^\top -\bar{\lambda}^\top A)x+\frac{1}{2}x^\top Qx+\rho w+\bar{\lambda}^\top b\\
\mathrm{s.t.}\ &(x,w)\in \cup_i(P_i\cap(\mathbb{R}^{n_1}\times \mathbb{Z}^{n_2}\times \mathbb{R})+\mathrm{int.cone}(R_i^J)\times\{0\})\\
&\|x\|_\infty\leq M\\
\geq \inf_{x,w}\ &(c^\top -\bar{\lambda}^\top A)x+\frac{1}{2}x^\top Qx+\rho w+\bar{\lambda}^\top b\\
\mathrm{s.t.}\ &(x,w)\in P^o,\ \|x\|_\infty\leq M.
\end{aligned}\]

Since
\[
\begin{aligned}
z_\rho^{\mathrm{LR+}}(\bar{\lambda})=\inf_{x,w}\ &(c^\top -\bar{\lambda}^\top A)x+\frac{1}{2}x^\top Qx+\rho w+\bar{\lambda}^\top b\\
\mathrm{s.t.}\ &(x,w)\in P^o\\
\leq \inf_{x,w}\ &(c^\top -\bar{\lambda}^\top A)x+\frac{1}{2}x^\top Qx+\rho w+\bar{\lambda}^\top b\\
\mathrm{s.t.}\ &(x,w)\in P^o,\ \|x\|_\infty\leq M,
\end{aligned}
\]
equality holds and the proof is completed.
\end{proof}

Next, we will generalize the result to any norm penalty and any dual variable.
\EPR*
\begin{proof}
Denote the $\rho^*(\bar{\lambda})$ in \cref{thm:fixeald} as $\hat{\rho}$ to avoid confusion.

As $\psi(\cdot)$ is a norm function, there exists $ \gamma\in[1,+\infty)$ such that $\gamma\|\cdot\|_\infty\geq \psi(\cdot)\geq\|\cdot\|_\infty/\gamma$. Without loss of generality, we round up $\gamma$ to a closest integer, which is still a constant decided only by $\|\cdot\|_\infty$ and $\psi(\cdot)$. Therefore, by letting $\rho^*(\bar{\lambda})=\gamma\hat{\rho}$, which still has small complexity, we have
\[
\begin{aligned}
z^\mathrm{LR+}_{\rho^*(\bar{\lambda})}(\bar{\lambda})=&\inf_{x\in X}\{c^\top x+\frac{1}{2}x^\top Q x+\bar{\lambda}^\top(b-Ax)+\rho^*(\bar{\lambda})\psi(b-Ax)\}\\
\geq&\inf_{x\in X}\{c^\top x+\frac{1}{2}x^\top Q x+\bar{\lambda}^\top(b-Ax)+\hat{\rho}\|b-Ax\|_\infty\}\\
=&z^\mathrm{IP},
\end{aligned}
\]
where the last equation comes from \cref{thm:fixeald}. Along with $z_\rho^{\mathrm{LR+}}(\lambda)\leq z_\rho^{\mathrm{LD+}}\leq z^\mathrm{IP}$, we have $z^\mathrm{LR+}_{\rho^*(\bar{\lambda})}(\bar{\lambda})=z^\mathrm{LD+}_{\rho^*(\bar{\lambda})}=z^\mathrm{IP}$ and (a) is proven. Now it only remains to show that we can replace $\bar{\lambda}$ by any dual vector $\tilde{\lambda}\in\mathbb{R}^m$.

From Cauchy-Schwarz inequality, we have
$$-\|\lambda\|_2\|b-Ax\|_2\leq\lambda^\top(b-Ax)\leq\|\lambda\|_2\|b-Ax\|_2.$$

Again, applying the property of the norm, there exists $ \eta\in[1,+\infty)\cap\mathbb{Z}_+$ decided only by $\|\cdot\|_2$ and $\psi(\cdot)$, such that $\eta\|\cdot\|_2\geq \psi(\cdot)\geq\|\cdot\|_2/\eta$, and we have
$$\tilde{\lambda}(b-Ax)-\bar{\lambda}^\top (b-Ax)\geq-\eta\|\tilde{\lambda}-\bar{\lambda}\|_2\psi(b-Ax).$$

By setting $\rho^*(\tilde{\lambda})=\lceil\rho^*(\bar{\lambda})+\eta\|\tilde{\lambda}-\bar{\lambda}\|_2\rceil$, which has $\mathcal{F},\tilde{\lambda}$-small complexity, we have
\[
\begin{aligned}
z^\mathrm{LR+}_{\rho^*(\tilde{\lambda})}(\tilde{\lambda})=&\inf_{x\in X}\{c^\top x+\frac{1}{2}x^\top Q x+\lambda^\top(b-Ax)+\rho^*(\tilde{\lambda})\psi(b-Ax)\}\\
\geq &\inf_{x\in X}\{c^\top x+\frac{1}{2}x^\top Q x+\lambda^\top(b-Ax)+\rho^*(\bar{\lambda})\psi(b-Ax)\}\\
=&z^\mathrm{LR+}_{\rho^*(\bar{\lambda})}(\bar{\lambda})=z^\mathrm{IP}.
\end{aligned}
\]

Therefore, along with $z_\rho^{\mathrm{LR+}}(\lambda)\leq z^\mathrm{IP}$, we have $z^\mathrm{LR+}_{\rho^*(\tilde{\lambda})}(\tilde{\lambda})=z^\mathrm{IP}$.
\end{proof}
\begin{remark}
The results also apply to MILP, which yield that exact penalty weight $\rho^*$ (which is detailedly discussed in \cite{feizollahi2017exact}) also has $\mathcal{F}$-small complexity.
\end{remark}
\section{Conclusions}
\label{sec:conclusions}
In this paper, we investigate ALD for MIQP. We prove that an asymptotic zero duality gap is reachable as the penalty weight goes to infinity, under some mild conditions (\cref{assum:penalty}) on the penalty function. We also show that a finite penalty weight is enough for an exact penalty when we use any norm as the penalty function. Moreover, we prove that a penalty weight of polynomial size is enough to give an exact penalty representative.

By dualizing and penalizing the difficult constraints using ALD, we can convert the problem to one with an easy feasible region, while maintains the optimality of the original optimal points. However, by introducing a penalty term in the objective function, it might become more difficult to deal with. In addition, as ALD does not deal with integer constraints, the problem is still far from convex.

A special case where the easy constraints are separable, leads us to consider the alternating direction method of multipliers (ADMM) \cite{boyd2011distributed} and relative update schemes, which are proposed to solve convex problems separably. However, for mixed integer problems, such methods are mainly heuristic, like \cite{takapoui2017simple} for MIQP based on ADMM. Future development of separable exact algorithms utilizing the strong duality results and solving general non-convex problems is a potential direction of research.


Asymptotic evaluation of $\int_0^\infty\left(\frac{\sin x}{x}\right)^n\;dx$

Abstract: We consider the integral $\int_0^\infty\left(\frac{\sin x}{x}\right)^n\;dx$ as a function of the positive integer $n$. We show that there exists an asymptotic series in $\frac{1}{n}$ and compute the first terms of this series together with an explicit error bound.

\section{Introduction and results}
In this note we show the following.
\begin{Theo}
\label{thm:series exists}
There exists an asymptotic series
\[
\int_0^\infty\left(\frac{\sin x}{x}\right)^n\;dx \approx \frac{\pi}{2}\left(1+\sum_{\nu=1}^\infty \frac{a_\nu}{n^\nu}\right)
\]
\end{Theo}
Here the series on the right is an asymptotic series in the sense of Poincar\'e, that is, we write $f(n)\approx\sum_{\nu=0}^\infty \frac{a_\nu}{n^\nu}$, if for every fixed $k$ we have $f(n)=\sum_{\nu=0}^k \frac{a_\nu}{n^\nu} + \mathcal{O}(n^{-k-1})$. We do not know whether the series is converging for $n$ sufficiently large, but we seriously doubt it. In fact, unless there is an unexpected amount of cancelation, the coefficients $a_\nu$ grow about as fast as $\nu!$.

Our main motivation for studying this integral is the fact that the intersection of the unit cube with a plane passing through the midpoint, which is orthogonal to a diagonal of the cube has $(n-1)$-dimensional measure equal to $\frac{2\sqrt{n}}{\pi}\int_0^\infty\left(\frac{\sin t}{t}\right)^n\;dt$. For an overview of this and related results we refer the reader to the work of Chakerian and Logothetti\cite{CL}. As the sum of $n$ independent random variables uniformly distributed on $[0,1]$ equals the $L^1$-norm of a random  point in the unit cube, these intersections occur naturally in probabilistic problems. For an example we refer the reader to the work of Silberstein\cite{Silberstein}.

For $n=1$ the integrand is the $\mathrm{sinc}$-function, which plays a crucial r\^ole in signal processing, as witnessed by the importance of the Nyquist-Shannon sampling theorem, for a historical overview we refer the reader to \cite{Luke}. After a suitable renormalisation, the function $\left(\frac{\sin x}{x}\right)^n$ is the Fourier transform of the $B$-spline of order $n$, which re also of importance in the theory of sampling. For an overview we refer the reader to the work of Butzer, Splettstößer and Stens\cite{BSS}, in particular section 4.1.

The coefficients $a_\nu$ can be computed, and the error in the approximation can be bounded effectively. As an example, we compute the following.
\begin{Prop}
\label{Prop:effective bound}
For $n\not\in\{2, 4, 6, 8, 10\}$ we have
\begin{multline*}
\int_0^\infty\left(\frac{\sin x}{x}\right)^n\;dx\\ = \sqrt{\frac{3\pi}{2n}}\left(
1-\frac{3}{20 n}-\frac{13}{1120 n^2}+\frac{27}{3200
   n^3}+\frac{52791}{3942400 n^4}+\widetilde{\mathcal{O}}\left(\frac{7.26\cdot 10^{-3}}{n^5}\right)\right)
\end{multline*}
\end{Prop}
Here $\widetilde{\mathcal{O}}$ denotes the effective Landau symbol, that is, the implied constant in the Landau symbol is of absolute value at most 1.

As an application we have the following, which answers a question by Schneider\cite{Schneider}.
\begin{Prop}
\label{Prop:Application}
For all $n\neq 4$ we have
\begin{equation}
\label{eq:Application}
2^{1-n}\cdot n\cdot\binom{n-1}{\lfloor(n-1)/2\rfloor}>\left(\frac{2}{\pi}\int\limits_0^\infty\left(\frac{\sin x}{x}\right)^n\;dx\right)^{-1}.
\end{equation}
\end{Prop}
Schneider\cite{Schneider} showed that (\ref{eq:Application}) holds for all large $n$ and stated that it probably holds for all $n\neq 4$. In particular, Proposition~\ref{Prop:Application} shows that the condition ``$n=3$ or $n$ sufficiently large'' in \cite[Theorem~2]{Schneider} can be replaced by $n\neq 4$.

All computations were performed using Mathematica 11.3.

\section{Existence of an asymptotic series}
We begin by restricting the relevant range of the integral.
\begin{Lem}
We have
\[
0\leq \int\limits_1^\infty\left(\frac{\sin x}{x}\right)^n\;dx \leq e^{-n/6}
\]
\end{Lem}
\begin{proof}
For $n=1, 2$ the claim follows by direct computation, hence, from now on we assume that $n\geq 3$. In particular, the integral converges absolutely.

The lower bound is trivial for $n$ even. For $n$ odd the integral consists of a positive integral over the range $[1, \frac{\pi}{2}]$, and a sequence of ranges of the form $[2k\pi+\frac{\pi}{2}, (2k+2)\pi+\frac{\pi}{2}]$. Considering a single interval of this form we have
\begin{multline*}
\int\limits_{2k\pi+\pi/2}^{2(k+1)\pi+\pi/2} \left(\frac{\sin x}{x}\right)^n\;dx =\\
 \int\limits_{\pi/2}^\pi\left(\frac{1}{(2k\pi+t)^n}-\frac{1}{((2k+2)\pi-t)^n} - \frac{1}{((2k+1)\pi+t)^n}+\frac{1}{(2k+3)\pi-t)^n}\right)\sin^n t,
\end{multline*}
and the integrand is positive as $\frac{1}{t^n}$ is convex. For the upper bound we split the integral into the range $[1, \frac{\pi}{2}]$ and $[\frac{\pi}{2}, \infty]$. For the first range we have
\[
\frac{\sin x}{x}\leq\sin 1=0.8414\dots<0.8464\dots=e^{1/6},
\]
hence, the contribution of this range is at most $(\frac{\pi}{2}-1)e^{-n/6}$. For the second part we use $\sin x\leq 1$, and see that the integral is bounded by $\int_{\pi/2}^\infty\frac{dx}{x^n} = \frac{1}{n-1}\left(\frac{\pi}{2}\right)^{n-1}$. Our claim now follows.
\end{proof}
\begin{Lem}
\label{Lem:f approx}
Let $\sum_{k\geq 1} a_k x^k$ be the Taylor series of $\log\frac{\sin x}{x}$. Then we have $|a_k|<0.517\cdot 3^{-k}$, and for $|x|<\frac{\pi}{2}$ and $K$ even the error bound
\[
\left|\log\frac{\sin x}{x} - \sum_{k=1}^K a_n x^n\right| < 1.09 \left(\frac{x}{3}\right)^{K+2}
\]
\end{Lem}
\begin{proof}
Put $f(z)=\log\frac{\sin z}{z}$. Note that $z$ is holomorphic for $|z|<\pi$ and symmetric. We have for $0<r<\pi$
\[
|a_k| = \left|\frac{f^{(k)}(0)}{n!}\right| = \frac{1}{2\pi}\left|\int\limits_{\partial B_r(0)}\frac{f(\zeta)}{(-\zeta)^{k+1}}\;d\zeta\right| \leq \frac{1}{2\pi r^{k+1}}\int\limits_{\partial B_r(0)} |f(\zeta)|\;d\zeta.
\]
We choose $r=3$ and compute the integral numerically to be $M=9.733\dots$, and obtain the claimed bound for $|a_k|$. We conclude that for $|x|\leq\frac{\pi}{2}$
\begin{multline*}
\left|\log\frac{\sin x}{x} - \sum_{k=1}^K a_k x^k\right| = \left|\sum_{k=K+2}^\infty a_k x^k\right| \leq 0.517\left(\frac{x}{3}\right)^{K+2}\sum_{k=0}^\infty \left(\frac{\pi}{6}\right)^k \leq 1.09 \left(\frac{x}{3}\right)^{K+2}
\end{multline*}
\end{proof}
\begin{Lem}
\label{Lem:f upper bound}
For $|x|\leq1$ we have
\[
\log\left(\frac{\sin x}{x}\right)\leq -\frac{x^2}{6}.
\]
\end{Lem}
\begin{proof}
We compute the first coefficients of the Taylor series as
\[
\log\left(\frac{\sin x}{x}\right) = -\frac{x^2}{6}-\frac{x^4}{180}-\frac{x^6}{2835} + \mathcal{O}(x^{8}).
\]
By Lemma~\ref{Lem:f approx} we obtain
\[
\log\left(\frac{\sin x}{x}\right) < -\frac{x^2}{6}-\frac{x^4}{180}-\frac{x^6}{2835} + 1.09\left(\frac{x}{3}\right)^{8}.
\]
For $x\leq 1$ the second term always dominates the last one, and our claim follows.
\end{proof}

By Lemma~\ref{Lem:f upper bound} we obtain
\[
\int\limits_{n^{-1/3}}^1\left(\frac{\sin x}{x}\right)^n\;dx \leq \int\limits_{n^{-1/3}}^1\exp\left(-\frac{nx^2}{6}\right)\;dx < e^{-n^{1/3}/6}
\]
We now compute
\begin{eqnarray*}
\int\limits_0^{n^{-1/3}}\left(\frac{\sin x}{x}\right)^n\;dx & = & \int\limits_0^{n^{-1/3}}\exp\left(n\log\left(\frac{\sin x}{x}\right)\right)\;dx\\  & = & \int\limits_0^{n^{-1/3}}\exp\left(-\frac{nx^2}{6} + n\sum_{k=2}^K a_{2k} x^{2k} + \mathcal{O}\left(nx^{2K+2}\right)  \right)\;dx\\
& = & \int\limits_0^{n^{-1/3}}\exp\left(-\frac{nx^2}{6} + n\sum_{k=2}^K a_{2k} x^{2k} \right)\;dx\\
&& + \int\limits_0^{n^{-1/3}}\exp\left(-\frac{nx^2}{6}\right)\left(e^{\mathcal{O}(nx^{2K+2})} -1 \right)\;dx\\
 & = & \int\limits_0^{\frac{n^{1/6}}{\sqrt{3}}} \exp\left(-\frac{u^2}{2} + \sum_{k=2}^K a_{2k} \frac{(3u^2)^k}{n^{k-1}} \right)\;\frac{du}{\sqrt{n/3}}\\
 && + \int\limits_0^{\frac{n^{1/6}}{\sqrt{3}}} e^{-\frac{u^2}{2}}\left(e^{\mathcal{O}(n^{-K}u^{2K+2})} -1 \right)\;\frac{du}{\sqrt{n/3}}.
\end{eqnarray*}
We first estimate the second integral. For $u<n^{1/6}$, the second factor of the integrand is $\mathcal{O}(n^{-\frac{2}{3}K+\frac{1}{3}})$, thus the integral is bounded by $\int_0^\infty e^{-\frac{u^2}{2}}\mathcal{O}(n^{-\frac{2}{3}K})\;du = \mathcal{O}(n^{-\frac{2}{3}K})$.

The first integral contributes to the main term. We have
\[
\exp\left(\sum_{k=2}^K a_{2k} \frac{(3u^2)^k}{n^{k-1}} \right) = 1+\sum_{k=2}^\infty\sum_{\frac{k}{2}\leq\ell\leq k-1}b_{k, \ell}^{(K)} \frac{u^{2k}}{n^\ell},
\]
where
\[
b_{k, \ell}^{(K)} = \frac{3^k}{(k-\ell)!} \underset{2\leq \kappa_i\leq K}{\sum_{\kappa_1+\dots\kappa_{k-\ell}=k}} \prod_{i=1}^{k-\ell}a_{2\kappa_i}\leq \frac{0.517^{k-\ell}}{3^k(k-\ell)!}\underset{2\leq \kappa_i\leq K}{\sum_{\kappa_1+\dots\kappa_{k-\ell}=k}} 1\leq \frac{0.517^{k-\ell}}{3^k(k-\ell)!}\binom{\ell-1}{k-\ell-1}.
\]
Here we have used the fact that
\[
\underset{2\leq \kappa_i\leq K}{\sum_{\kappa_1+\dots\kappa_{k-\ell}=k}} 1\leq\underset{2\leq \kappa_i}{\sum_{\kappa_1+\dots\kappa_{k-\ell}=k}} 1 = \binom{\ell-1}{k-\ell-1}.
\]
To bound sums involving $b_{k, \ell}$, the following is helpful.
\begin{Lem}
\label{Lem:Fibonacci}
We have $\sum_{k/2\leq\ell\leq k-1}\binom{\ell-1}{k-\ell-1} = F_{k-1}$, where $F_n$ denotes the $n$-th Fibonacci number.
\end{Lem}
\begin{proof}
We prove our claim by induction on $k$. For $k\leq 3$ the statement is immediate.
Note that $\sum_{k/2\leq\ell\leq k-1}\binom{\ell-1}{k-\ell-1}$ equals the number of possibilities to write $k$ as a sum of integers $\kappa_i\geq 2$. Each such sum either ends with the summand 2, or it ends with a summand $>2$. The number of sums of the first kind equals the number of representations of $k-2$, as we can delete the last summand. The number of sums of the second kind equals the number of representations of $k-1$, as we can reduce the last summand by 1. Hence our claim follows.
\end{proof}
Using this notation the first integral is
\begin{multline*}
\int\limits_0^{\frac{n^{1/6}}{\sqrt{3}}} \exp\left(-\frac{u^2}{2} + \sum_{k=2}^K a_{2k} \frac{(3u^2)^k}{n^{k-1}} \right)\;\frac{du}{\sqrt{n/3}}\\
 = \int\limits_0^{\frac{n^{1/6}}{\sqrt{3}}} \exp\left(-\frac{u^2}{2}\right)\left(1+\sum_{k=2}^\infty\sum_{\frac{k}{2}\leq\ell\leq k-1}b_{k, \ell}^{(K)} \frac{u^{2k}}{n^\ell}\right)\;du\\
= \int\limits_0^{\frac{n^{1/6}}{\sqrt{3}}} \exp\left(-\frac{u^2}{2}\right)\left(1+\sum_{k=2}^K\sum_{\frac{k}{2}\leq\ell\leq k-1}b_{k, \ell}^{(K)} \frac{u^{2k}}{n^\ell}\right)\;du\\
 + \mathcal{O}\left(n^{-K/6}\int\limits_0^{\frac{n^{1/6}}{\sqrt{3}}} \exp\left(-\frac{u^2}{2}\right)\left(\sum_{k=K+1}^\infty\sum_{\frac{k}{2}\leq\ell\leq k-1}b_{k, \ell}^{(K)}\right)\;du\right).
\end{multline*}
We have
\begin{multline*}
\sum_{k=K+1}^\infty\sum_{\frac{k}{2}\leq\ell\leq k-1}b_{k, \ell}^{(K)} \leq \sum_{k=K+1}^\infty\sum_{\frac{k}{2}\leq\ell\leq k-1}\frac{0.517^{k-\ell}}{3^k(k-\ell)!}\binom{\ell-1}{k-\ell-1}\\
 \leq \sum_{k=K+1} \frac{1}{3^k} \sum_{\frac{k}{2}\leq\ell\leq k-1}\binom{\ell-1}{k-\ell-1} = \sum_{k={K+1}}^\infty\frac{F_{k-1}}{3^k}<1
\end{multline*}
As both the sum and the integral in the error term converge absolutely, the error is $\mathcal{O}(n^{-K/6})$. For the main term we interchange sum and integral, and extend the integral to $[0, \infty)$ to obtain
\begin{multline*}
\int\limits_0^{\frac{n^{1/6}}{\sqrt{3}}} \exp\left(-\frac{u^2}{2}\right)\left(1+\sum_{k=2}^K\sum_{\frac{k}{2}\leq\ell\leq k-1}b_{k, \ell}^{(K)} \frac{u^{2k}}{n^\ell}\right)\;du\\
 = \int\limits_0^{\frac{n^{1/6}}{\sqrt{3}}} \exp\left(-\frac{u^2}{2}\right)\;du + \sum_{k=2}^K\sum_{\frac{k}{2}\leq\ell\leq k-1}\frac{b_{k, \ell}^{(K)}}{n^\ell}\int\limits_0^{\frac{n^{1/6}}{\sqrt{3}}}\exp\left(-\frac{u^2}{2}\right)u^{2k}\;du\\
  = \int\limits_0^\infty \exp\left(-\frac{u^2}{2}\right)\;du + \sum_{k=2}^K\sum_{\frac{k}{2}\leq\ell\leq k-1}\frac{b_{k, \ell}^{(K)}}{n^\ell}\int\limits_0^\infty\exp\left(-\frac{u^2}{2}\right)u^{2k}\;du+\mathcal{O}\left(n^{\frac{2K}{3}}e^{-n^{1/3}}\right)\\
   = \sqrt{\frac{\pi}{2}}\left(1+\sum_{\ell=1}^{K-1}\frac{1}{n^\ell}\sum_{k=\ell+1}^{2\ell} b_{k,\ell}^K(2k-1)!!\right)+\mathcal{O}\left(n^{\frac{2K}{3}}e^{-n^{1/3}}\right),
\end{multline*}
where in the last step we used the fact that the moments of the normal distribution are given by
\[
\int\limits_0^\infty e^{-\frac{u^2}{2}}u^{2k}\;du = (2k-1)!!\sqrt{\frac{\pi}{2}},
\]
where $(2n+1)!!=1\cdot3\cdot 5\cdots (2n-1)(2n+1)$ is the double factorial, see e.g. \cite[7.4.4]{AS}.

The existence of an asymptotic series now follows.

\section{Explicite computations}

The explicite computation is similar to the asymptotic approach. However, a term of magnitude $e^{-n^{1/3}/6}$, which is negligible for large $n$, would complicate matters a lot for medium $n$. On the other hand, we can easily compute the Taylor series of $\log\left(\frac{\sin x}{x}\right)$, and the real coefficients are significantly smaller than Lemma~\ref{Lem:f approx} predicts. Therefore it is advantageous to choose different parameters. In particular we will compute higher order terms even if they are negligible for large $n$. Explicitly computing the Taylor series up to $x^{10}$ and estimating the remainder using Lemma~\ref{Lem:f approx}, we see that for $x\leq\frac{\pi}{2}$ we have
\[
0\geq \log\frac{\sin x}{x} - P(x) \geq - 1.4\cdot 10^{-12} x^{22},
\]
where $P(x)=\sum_{k=1}^{10} a_k x^{2k}$. Combining this estimate with Lemma~\ref{Lem:f upper bound} and the fact that for $\delta>0$
\[
\min(1, e^\delta-1) \leq \delta \max_{\delta_\in[0, \log 2} \frac{e^\delta-1}{\delta}= \frac{\delta}{\log 2}
\]
 we obtain
\begin{eqnarray*}
\int\limits_0^{\pi/2}\left(\frac{\sin x}{x}\right)^n\;dx & = & \int\limits_0^{\pi/2}\exp\left(nP(x)\right)\;dx\\
&& + \widetilde{\mathcal{O}}\left(2.1\cdot 10^{-12} n\int\limits_0^{\pi/2}e^{-n\frac{x^2}{6}}x^{22}\;dx\right)\\
 & = & \int\limits_0^{\pi/2}\exp\left(nP(x)\right)\;dx\\
 &&+\widetilde{\mathcal{O}}\left(2.1\cdot 10^{-12} 3^{23/2} n^{-23/2}\sqrt{\frac{\pi}{2}}21!!\right)\\
 & = & \int\limits_0^{\pi/2}\exp\left(nP(x)\right)\;dx+\widetilde{\mathcal{O}}\left(11104 n^{-23/2}\right)
\end{eqnarray*}
We have
\[
\int\limits_0^{\pi/2}\exp\left(nP(x)\right)\;dx = \sqrt{\frac{3}{n}}\int\limits_0^{\frac{\pi\sqrt{n}}{2\sqrt{3}}} e^{-\frac{u^2}{2}}\left(1+\sum_{k\geq 2}\sum_{\frac{k}{2}\leq\ell\leq k-1}\frac{b_{k,\ell}^{(10)}}{n^\ell}u^{2k}\right)\;du
\]
Extracting the terms with $\ell\leq 10$ and extending the integral to $[0, \infty)$ we obtain
\begin{multline*}
\sqrt{\frac{3\pi}{2n}}\left(
1-\frac{3}{20 n}-\frac{13}{1120 n^2}+\frac{27}{3200
   n^3}+\frac{52791}{3942400 n^4}+\frac{482427}{66560000 n^5}\right.\\
   -\frac{124996631}{10035200000
   n^6}
   -\frac{5270328789}{136478720000 n^7}-\frac{7479063506161}{268461670400000
   n^8}\\
   \left.+\frac{6921977624613}{56518246400000 n^9} + \frac{2631854096507395099467}{1028632084480000000
   n^{10}}\right).
\end{multline*}
 We have
\[
\left(\frac{u+1}{u}\right)^{20}e^{-\frac{(u+1)^2}{2}+\frac{u^2}{2}} \leq e^{\frac{20}{u}-u- \frac{1}{2}},
\]
hence, for $u_0>5$ and $k\leq 20$ we obtain
\[
\int\limits_{u_0}^\infty e^{-\frac{u^2}{2}}u^k\;du\leq \frac{e^{-\frac{u_0^2}{2}}u_0^k}{1-e^{-3/2}},
\]
and we conclude that the error introduced in the extension of the integral is bounded by
\begin{multline*}
\sqrt{\frac{3}{n}}\frac{e^{-\frac{\pi n}{12}}}{1-e^{-3/2}}\left(1+\sum_{2\leq k\leq 10}\sum_{\frac{k}{2}\leq\ell\leq k-1}\frac{|b_{k,\ell}^{(10)}|}{n^\ell}\left(\frac{\pi^2n}{12}\right)^k\right)\\
\leq 2.23\cdot e^{-\frac{\pi n}{12}}\left(4.04\cdot 10^{-2}n + 8.14\cdot 10^{-4} n^2+1.1\cdot 10^{-5} n^3+1.04\cdot 10^{-7}n^4+3.69\cdot 10^{-10}n^5\right)\\
\leq 1.59\cdot 10^{-9} n^5 e^{-\frac{\pi n}{12}} < n^{-\frac{23}{2}},
\end{multline*}
provided that $n\geq 400$. We conclude that for $n\geq 400$ we have
\begin{eqnarray}
\int\limits_0^\infty\left(\frac{\sin x}{x}\right)^n\;dx & = & \sqrt{\frac{3\pi}{2n}}\left(
1-\frac{3}{20 n}-\frac{13}{1120 n^2}+\frac{27}{3200
   n^3}+\frac{52791}{3942400 n^4}+\widetilde{\mathcal{O}}\left(\frac{7.25\cdot 10^{-3}}{n^5}\right)\right)\nonumber\\
\label{eq:almost there}   &&+\widetilde{\mathcal{O}}(e^{-n/6}) + \widetilde{\mathcal{O}}\left(\sqrt{\frac{3}{n}}\int\limits_0^{\frac{\pi\sqrt{n}}{2\sqrt{3}}} e^{-\frac{u^2}{2}}\sum_{k\geq 2}\sum_{\max(11, \frac{k}{2})\leq\ell\leq k-1}\frac{b_{k,\ell}^{(10)}}{n^\ell}u^{2k}\;du\right)
\end{eqnarray}
Next we bound the contribution of a summand with $\ell$ large. As $e^{-\frac{u^2}{2}}u^{2k}$ increases for $u<\sqrt{2k}$ and decreases for $u>2k$, we have the bound
\[
\int\limits_0^{\frac{\pi\sqrt{n}}{2\sqrt{3}}} e^{-\frac{u^2}{2}}u^{2k}\;du\leq\begin{cases}
(2k-1)!!, & k\leq \frac{\pi^2 n}{24},\\
\left(\frac{\pi\sqrt{n}}{2\sqrt{3}}\right)^{2k+1} e^{-\frac{\pi^2 n}{24}}, & k>\frac{\pi^2 n}{24}.
\end{cases}
\]
Hence, the contribution of the range $k>\frac{\pi^2 n}{24}$ is at most
\begin{multline*}
\sum_{k>\frac{\pi^2 n}{24}} \left(\frac{\pi\sqrt{n}}{2\sqrt{3}}\right)^{2k+1} e^{-\frac{\pi^2 n}{24}}\sum_{k/2\leq\ell\leq k-1} \frac{|b_{k, \ell}|}{n^\ell}\\
 \leq \sqrt{n}e^{-\frac{\pi^2 n}{24}}\sum_{k>\frac{\pi^2 n}{24}}\sum_{k/2\leq\ell\leq k-1} \frac{0.907^{2k+1} (0.517n)^{k-\ell}}{3^k(k-\ell)!}\binom{\ell-1}{k-\ell-1}\\
 \leq \sqrt{n}e^{-\frac{\pi^2 n}{24}+0.517 n}\sum_{k>\frac{\pi^2 n}{24}}\frac{0.907^{2k+1}F_{k-1}}{3^k} < 1.64\sqrt{n}\cdot 0.796^n.
\end{multline*}
The contribution of the summands belonging to a single $k\leq\frac{\pi^2 n}{24}$ is at most
\begin{multline*}
(2k-1)!!\sum_{\frac{k}{2}\leq\ell\leq k-1} \frac{b_{k,\ell}}{n^\ell} \leq \frac{(2k)!}{2^k k!}\sum_{\frac{k}{2}\leq\ell\leq k-1}\frac{ 0.517^{k-\ell}}{3^k(k-\ell)! n^\ell}\binom{\ell-1}{k-\ell-1}\\
\leq \frac{F_{k-1}(2k)!}{6^k k!}\sum_{\frac{k}{2}\leq\ell\leq k-1}\frac{ 0.517^{k-\ell}}{(k-\ell)! n^\ell} \leq 2\frac{F_{k-1}(2k)!}{6^k k!}\cdot\frac{0.517^{k/2}}{(k/2)!n^{k/2}} < 0.479^k\left(\frac{k}{n}\right)^{k/2}.
\end{multline*}
Hence, the sum over all $k\geq K$ is bounded by $1.45\cdot 0.479^K\left(\frac{K}{n}\right)^{K/2}$. We take $K=35$.

It remains to bound the range $\ell\geq 11$, $k\leq 34$. Here we obtain by direct computation
\[
\sum_{k\leq 34} \sum_{\max(11,\frac{k}{2})\leq\ell\leq k-1}\frac{b_{k, \ell}}{n^\ell}\leq \frac{5.5\cdot 10^7}{n^{11}},
\]
provided that $n>400$. We see that the last two error terms in (\ref{eq:almost there}) are bounded by
\[
e^{-n/6} + 1.45\cdot 0.479^{35}\left(\frac{35}{n}\right)^{\frac{35}{2}} + \frac{5.5\cdot 10^7}{n^{11}},
\]
which for $n\geq 400$ is bounded by $\frac{10^{-5}}{n^5}$, hence our claim follows in this range.

Finally for $n\leq 400$ we check Proposition~\ref{Prop:effective bound} directly using the following result due to
Chakerian and Logothetti \cite{CL}.
\begin{Lem}
We have
\[
\int\limits_0^\infty\left(\frac{\sin x}{x}\right)^n\;dx = \frac{\pi}{2^n(n-1)!}\sum_{j=0}^{\lfloor n/2\rfloor} (-1)^j\binom{n}{j}(n-2j)^{n-1}.
\]
\end{Lem}

\section{Proof of Proposition~\ref{Prop:Application}}

Note first that for all $n\geq 1$ we have that the $n^{-3}$ term in the asymptotic series in Proposition~\ref{Prop:effective bound} dominates the error term, hence, for all $n$ the right hand side of (\ref{eq:Application}) is at most
\[
\sqrt{\frac{\pi n}{6}}\left(1-\frac{3}{20n}-\frac{13}{1120n^2}\right)^{-1}.
\]
To estimate the left hand side we use Stirling's formula in the following form, see \cite[6.1.38]{AS}
\begin{Lem}
We have
\[
n! = \left(\frac{n}{e}\right)^n\sqrt{2\pi n} e^{\widetilde{\mathcal{O}}\left(\frac{1}{12 n}\right)}.
\]
\end{Lem}
We conclude that
\[
\binom{2m}{m} = \frac{2m!}{m!^2} = \frac{2^{2m}}{\sqrt{\pi m}}e^{\widetilde{\mathcal{O}}\left(\frac{1}{6m}\right)}
\]
and
\[
\binom{2m+1}{m}=\frac{1}{2}\binom{2m+2}{m+1} = \frac{2^{2m+1}}{\sqrt{\pi (m+1)}}e^{\widetilde{\mathcal{O}}\left(\frac{1}{6(m+1)}\right)}.
\]
We obtain that the left hand side of (\ref{eq:Application}) is
\[
\frac{n}{\sqrt{\pi\lfloor n/2\rfloor}}e^{\widetilde{\mathcal{O}}\left(\frac{1}{3n}\right)}\geq \sqrt{\frac{2n}{\pi}}\left(1-\frac{1}{3n}\right)
\]
Hence, to prove Proposition~\ref{Prop:Application} it suffices to check that Proposition~\ref{Prop:effective bound} holds, and that
\[
\sqrt{\frac{2n}{\pi}}\left(1-\frac{1}{3n}\right) > \sqrt{\frac{\pi n}{6}}\left(1-\frac{3}{20n}-\frac{13}{1120n^2}\right)^{-1},
\]
that is,
\[
\left(1-\frac{1}{3n}\right)\left(1-\frac{3}{20n}-\frac{13}{1120n^2}\right) > \sqrt{\frac{\pi^2}{12}} = 0.9068\ldots,
\]
and we see that this inequality holds for $n\geq 6$. We conclude that Proposition~\ref{Prop:Application} holds for $n\not\in\{1,2,3,4,5,6,8,10\}$, and by direct inspection we find that it holds for all $n\neq 4$.