Probabilistic Analysis of Linear Programming Decoding
We initiate the probabilistic analysis of linear programming (LP) decoding of low-density parity-check (LDPC) codes. Specifically, we show that for a random LDPC code ensemble, the linear programming decoder of Feldman et al. succeeds in correcting a constant fraction of errors with high probability. The fraction of correctable errors guaranteed by our analysis surpasses previous non-asymptotic results for LDPC codes, and in particular exceeds the best previous finite-length result on LP decoding by a factor greater than ten. This improvement stems in part from our analysis of probabilistic bit-flipping channels, as opposed to adversarial channels. At the core of our analysis is a novel combinatorial characterization of LP decoding success, based on the notion of a generalized matching. An interesting by-product of our analysis is to establish the existence of ``probabilistic expansion'' in random bipartite graphs, in which one requires only that almost every (as opposed to every) set of a certain size expands, for sets much larger than in the classical worst-case setting.
\subsection{#1}}
\newcommand{\ensuremath{\mathcal{C}}}{\ensuremath{\mathcal{C}}}
\newcommand{\ensuremath{\widehat{x}_{\operatorname{OPT}}}}{\ensuremath{\widehat{x}_{\operatorname{OPT}}}}
\newcommand{\ensuremath{\widehat{x}_{\operatorname{LP}}}}{\ensuremath{\widehat{x}_{\operatorname{LP}}}}
\newcommand{\ensuremath{\operatorname{conv}}}{\ensuremath{\operatorname{conv}}}
\newcommand{\ensuremath{\widehat{y}_{\operatorname{ML}}}}{\ensuremath{\widehat{y}_{\operatorname{ML}}}}
\newcommand{\ensuremath{\widehat{y}_{\operatorname{LP}}}}{\ensuremath{\widehat{y}_{\operatorname{LP}}}}

\newcommand{\ensuremath{\mathcal{G}}}{\ensuremath{\mathcal{G}}}

\newcommand{\ensuremath{Y}}{\ensuremath{Y}}
\newcommand{\ensuremath{y}}{\ensuremath{y}}
\newcommand{\ensuremath{\vec{\myreq}}}{\ensuremath{y}}

\newcommand{\ensuremath{\widebar{\myreq}}}{\ensuremath{\widebar{\ensuremath{y}}}}
\newcommand{\ensuremath{\widebar{\myreq}}}{\ensuremath{\widebar{\ensuremath{y}}}}


\newlength{\widebarargwidth}
\newlength{\widebarargheight}
\newlength{\widebarargdepth}
\DeclareRobustCommand{\widebar}[1]{%
  \settowidth{\widebarargwidth}{\ensuremath{#1}}%
  \settoheight{\widebarargheight}{\ensuremath{#1}}%
  \settodepth{\widebarargdepth}{\ensuremath{#1}}%
  \addtolength{\widebarargwidth}{-0.3\widebarargheight}%
  \addtolength{\widebarargwidth}{-0.3\widebarargdepth}%
  \makebox[0pt][l]{\hspace{0.3\widebarargheight}%
    \hspace{0.3\widebarargdepth}%
    \addtolength{\widebarargheight}{0.3ex}%
    \rule[\widebarargheight]{0.95\widebarargwidth}{0.1ex}}%
  {#1}}


\makeatletter
\def\@cite#1#2{[\if@tempswa #2 \fi #1]}
\makeatother


\makeatletter
\long\def\@makecaption#1#2{
        \vskip 0.8ex
        \setbox\@tempboxa\hbox{\small {\bf #1:} #2}
        \parindent 1.5em 
        \dimen0=\hsize
        \advance\dimen0 by -3em
        \ifdim \wd\@tempboxa >\dimen0
                \hbox to \hsize{
                        \parindent 0em
                        \hfil 
                        \parbox{\dimen0}{\def\baselinestretch{0.96}\small
                                {\bf #1.} #2
                               
                                } 
                        \hfil}
        \else \hbox to \hsize{\hfil \box\@tempboxa \hfil}
        \fi
        }
\makeatother



\begin{document}




\title{Probabilistic Analysis of Linear Programming Decoding}
\author{ ~~~~~~~~~~Constantinos Daskalakis$^1$,\thanks{CD supported by NSF
Grant CCF-0515259; part of this work was done while the author was
visiting Microsoft Research.}
\and Alexandros G. Dimakis$^1$,\thanks{AGD supported by NSF Grants
DMS-0528488, CCF-0635372, and an UC MICRO grant with Marvell
Corporation; part of this work was done while the author was visiting
EPFL.}
\and
\and Richard M. Karp$^1$,\thanks{RMK supported by NSF Grant
CCF-0515259.}
\and and Martin J. Wainwright$^{1,2}$\thanks{MOW supported by an
Alfred P. Sloan Foundation Fellowship, NSF grants DMS-0528488,
CCF-0635372, and an UC MICRO Grant with Marvell Corporation.}}


\maketitle


\begin{abstract}
We initiate the probabilistic analysis of linear programming (LP)
decoding of low-density parity-check (LDPC) codes. Specifically, we
show that for a random LDPC code ensemble, the linear programming
decoder of Feldman et al. succeeds in correcting a constant fraction
of errors with high probability. The fraction of correctable errors
guaranteed by our analysis surpasses previous non-asymptotic results
for LDPC codes, and in particular exceeds the best previous
finite-length result on LP decoding by a factor greater than ten. This
improvement stems in part from our analysis of probabilistic
bit-flipping channels, as opposed to adversarial channels.  At the
core of our analysis is a novel combinatorial characterization of LP
decoding success, based on the notion of a generalized matching.  An
interesting by-product of our analysis is to establish the existence
of ``probabilistic expansion'' in random bipartite graphs, in which one
requires only that almost every (as opposed to every) set of a certain
size expands, for sets much larger than in the classical
worst-case setting.
\end{abstract}
\noindent {\bf{Keywords:}} Error-control coding; channel coding;
binary symmetric channel; factor graphs; sum-product algorithm; linear
programming decoding; low-density parity check codes; randomized
algorithms; expanders.




\section{Introduction} \label{sec:introduction}



Low-density parity-check (LDPC) codes are a class of sparse binary
linear codes, first introduced by Gallager~\cite{Gallager63}, and
subsequently studied extensively by various
researchers~\cite{Mackay99,Richardson01a,Luby01}. 
See the book by Richardson and Urbanke~\cite{mct} for a comprehensive treatment of the subject. 
When decoded
with efficient iterative algorithms (e.g., the sum-product
algorithm~\cite{Frank01}), suitably designed classes of LDPC codes
yield error-correcting performance extremely close to the Shannon
capacity of noisy channels for very large codes~\cite{Chung01}.  Most
extant methods for analyzing the performance of iterative decoding
algorithms for LDPC codes---notably the method of density
evolution~\cite{Luby01,Richardson01a}---are asymptotic in nature,
based on exploiting the high girth of very large random graphs.
Therefore, the thresholds computed using density evolution are only
estimates of the true algorithm behavior, since they assume a
cycle-free message history.  In fact, the predictions of such methods
are well-known to be inaccurate for specific codes of intermediate
block length (e.g., codes with a few hundreds or thousands of
bits). For this reason, our current understanding of practical
decoders for smaller codes, which are required for applications with
delay constraints, is relatively limited.

The focus of this paper is the probabilistic analysis of linear
programming (LP) decoding, a technique first introduced by Feldman
et al.~\cite{Feldman02,Feldman05} as an alternative to iterative
algorithms for decoding LDPC codes.  The underlying idea is a
standard one in combinatorial optimization---namely, to solve a
particular \emph{linear programming} (LP) relaxation of the
integer program corresponding to maximum likelihood (optimal)
decoding.
Although the practical performance of LP decoding is comparable to
message-passing decoding, a significant advantage is its relative
amenability to non-asymptotic analysis.  Moreover, there turn out to
be a number of important theoretical connections between the LP
decoding and standard forms of iterative
decoding~\cite{KoeVon03,WaiJaaWil05b}. These connections allow
theoretical insight from the LP decoding perspective to be transferred
to iterative decoding algorithms.



\myparagraph{Previous work} The technique of LP decoding was
introduced for turbo-like codes~\cite{Feldman02}, extended to LDPC
codes~\cite{FelKarWai02,Feldman05}, and further studied by various
researchers (e.g.,~\cite{VonKoe06,CheSte06,
Feldman05b,DimWai06,lpcap_soda,Halabi05}).
Significant recent interest has focused on post-processing algorithms that use the ML-certificate property of LP decoding to achieve near ML performance (see \cite{DimWai06, ChertkovEF}) and also
\cite{SDmixed07, Taghavi07}).

  For concatenated expander
codes, Feldman and Stein~\cite{lpcap_soda} showed that LP decoding can
achieve capacity; see also~\cite{bargzemor2, Kelley07}. For the standard LDPC
codes used in practice, the best positive result from previous
work~\cite{Feldman05b} is the existence of a constant
$\beta > 0$, depending on the rate of the code, such that LP decoding
can correct \emph{any} bit-flipping pattern consisting of at most
$\beta \ensuremath{n}$ bit flips.  (In short, we say that LP decoding can
correct a $\beta$-fraction of errors.)  As a concrete example, for
suitable classes of rate $1/2$ LDPC codes, Feldman et
al.~\cite{Feldman05b} established that $\beta = 0.000177$ is
achievable.  However, this analysis~\cite{Feldman05b} was worst-case
in nature, essentially assuming an adversarial channel model.  Such
analysis yields overly conservative predictions for the probabilistic
channel models that are of more practical interest.  Consequently, an
important direction---and the goal of this paper---is to develop
methods for finite-length and \emph{average-case analysis} of the LP
decoding method.

\myparagraph{Our contributions} This paper initiates the average-case
analysis of LP decoding for LDPC codes.
In particular, we analyze the following question: what is the
probability, given that a random subset of $\alpha \ensuremath{n}$ bits is
flipped by the channel, that LP decoding succeeds in recovering
correctly the transmitted codeword?  As a concrete example, we prove
that for bit-degree-regular LDPC codes of rate $1/2$ and a random
error pattern with $\alpha \ensuremath{n}$ bit flips, LP decoding will
recover the correct codeword, with probability converging to
one\footnote{Note that our analysis yields a bound on the probability
of failure for every finite block length $n$.}, for all $\alpha$ up to
at least $0.002$. This guarantee is roughly ten times higher than the
best guarantee from prior work~\cite{Feldman05b}, derived in the
setting of an adversarial channel.  Our proof is based on analyzing
the dual of the decoding linear program and obtaining a simple
graph-theoretic condition for certifying a zero-valued solution of the
dual LP, which (by strong duality) ensures that the LP decoder
recovers the transmitted codeword.  We show that this dual witness has
an intuitive interpretation in terms of the existence of hyperflow
from the flipped to the unflipped bits.  Although this paper focuses
exclusively on the binary symmetric channel (BSC), the poison hyperflow
is an exact characterization of LP decoding for any
memoryless binary input symmetric output (MBIOS) channel. 
 We then
show that such a hyperflow witness exists with high probability under
random errors in the bit-degree-regular LDPC ensemble.  The argument
itself entails a fairly delicate sequence of union bounds and
concentration inequalities, exploiting expansion and matchings on
random bipartite graphs.
 

\myparagraph{Probabilistic Expanders} 

An interesting by-product of our analysis is
the proof of the existence of \emph{probabilistic expanders}---that is,
bipartite graphs in which \emph{almost all sets} of vertices of
size up to $\alpha \ensuremath{n}$ \emph{and their subsets} have large expansion. 
One key point is that it is not sufficient to require a random subset of vertices to expand w.h.p., 
because we use the expansion combined with Hall's theorem to guarantee large matchings. What we need instead is that a random subset of vertices and all its subsets will expand w.h.p. 
which by Hall's theorem will guarantee that a random subset will have a matching. 
In effect, by relaxing the expansion requirement from every
set to almost all sets of a given size, we show that one can obtain
much larger expansion factors, and corresponding stronger guarantees
on error correction.
Our analysis relies on the fact that a random bipartite graph, conditioned on all the small sets having some expansion, will also have this probabilistic expansion for much larger constants $\alpha$.  
 This innovation allows us to go beyond the
worst-case fraction of errors guaranteed by traditional expansion
arguments~\cite{Feldman05b,ZipSpi96}.



The remainder of the paper is organized as follows. We begin in
Section~\ref{SecBackground} with background on error-control coding
and low-density parity-check codes, as well as the method of linear
programming (LP) decoding.  Section~\ref{SecMain} describes our main
result and Section~\ref{SecProof} provides the proof in a series of
lemmas, with more technical details deferred to the appendix.  We
conclude in Section~\ref{SecDiscussion} with a discussion.

\section{Background and Problem Formulation}
\label{SecBackground}

We begin with some background on low-density parity-check codes.
We then describe the LP decoding method, and formulate the problem
to be studied in this paper.

\myparagraph{Low-density parity-check codes} The purpose of an
error-correcting code is to introduce redundancy into a data sequence
so as to achieve error-free communication over a noisy channel.  Given
a binary vector of length $k$ (representing information to be
conveyed), the encoder maps it onto a codeword, corresponding to a
binary vector of length $n > k$.  The code rate is given by
$\ensuremath{R}=k/n$, corresponding to the ratio of information bits to
transmitted bits.  In a binary linear code, the set of all possible
codewords corresponds to a subspace of $\{0,1\}^n$, with a total of
$2^k$ elements (one for each possible information sequence).  The
codeword is then transmitted over a noisy channel.  In this paper, we
focus on the \emph{binary symmetric channel} (BSC), in which each bit
is flipped independently with probability $\alpha$.  Given the
received sequence from the channel, the goal of the decoder is to
correctly reconstruct the transmitted codeword (and hence the
underlying information sequence).

Any binary linear code can be described as the null space of a parity
check matrix $H \in \{0,1\}^{(n-k) \times n}$; more concretely, the
code $\ensuremath{\mathcal{C}}$ is given by the set of all binary strings $x \in
\{0,1\}^n$ such that $H x = 0$ in modulo two arithmetic.  A convenient
graphical representation of such a binary linear code is in terms of
its factor graph~\cite{Frank01}.  The factor graph associated with a
code $\ensuremath{\mathcal{C}}$ is a bipartite graph $\Graph = (\ensuremath{V}, \ensuremath{C})$,
with $\ensuremath{n} = |\ensuremath{V}|$ variable nodes corresponding to the
codeword bits (columns of the matrix $H$), and $\ensuremath{m} = n - k =
|\ensuremath{C}|$ nodes corresponding to the parity checks (rows of the
matrix $H$). Edges in the factor graph connect each variable node to
the parity checks that constrain it, so that the parity check matrix
$H$ specifies the adjacency matrix of the graph.  A \emph{low-density
parity-check} code is a binary linear code that can be expressed with
a sparse factor graph (i.e. one with $\Theta(1)$ edges per row).

Given a received sequence $\hat{y} \in \{0,1\}^n$ from the BSC, the
maximum likelihood (ML) decoding problem is to determine the closest
codeword (in Hamming distance).  It is well known that ML decoding for
general binary linear codes is NP-hard~\cite{Berlekamp78}, which
motivates the study of sub-optimal but practical algorithms for
decoding.


\myparagraph{LP decoding} \label{SecLPdecode}


We now describe how the problem of optimal decoding can be
reformulated as a linear program over the \emph{codeword polytope},
i.e. the convex hull of all codewords of the code $\ensuremath{\mathcal{C}}$. For every
bit $\hat{y}_i$ of the received sequence $\hat{y}$, define its
log-likelihood as $ \gamma_i= \log{\left( \frac{ \ensuremath{\mathbb{P}} [ \hat{y_i}
|y_i=0]}{ \ensuremath{\mathbb{P}} [ \hat{y_i} | y_i=1]}\right)}$, where $y_i$
represents the corresponding bit of the transmitted codeword $y$.
Using the memoryless property of the channel,
it can be seen that the maximum likelihood (ML) codeword is
\begin{equation}
\ensuremath{\widehat{y}_{\operatorname{ML}}} \, = \, \arg \min_{y \in \code} \sum _{i=1}^{n} \gamma_i y_i.
\end{equation}
Without changing the outcome of the maximization, we can replace the
code $\code$ by its convex hull $\operatorname{conv}(\code)$, and thus
express ML decoding as the linear program 
\begin{eqnarray}
\label{LP1}
\ensuremath{\widehat{y}_{\operatorname{ML}}} & = & \arg \min_{y \in \text{conv}(\code)} \sum _{i=1}^{n}
\gamma_i y_i.
\end{eqnarray}
Although we have converted the decoding problem from an integer
program to a linear program, it remains intractable because for
general factor graphs with cycles, the codeword polytope does not have
a concise description.

A natural approach, and one that is standard in operations research
and polyhedral combinatorics, is to relax the linear program by taking
only a polynomial set of constraints that approximate the codeword
polytope $\ensuremath{\operatorname{conv}}(\ensuremath{\mathcal{C}})$.  The first-order LP decoding
method~\cite{Feldman05} makes use of a relaxation that results from
looking at each parity check, or equivalently at each row of $H$, in
an independent manner. For each check $\ensuremath{a} \in \ensuremath{C}$ in the
code, denote by $\ensuremath{\mathcal{C}}_\ensuremath{a}$ the set of binary sequences that
satisfy it---that is, $\ensuremath{\mathcal{C}}_\ensuremath{a}$ corresponds to the local
parity check subcode defined by check $\ensuremath{a}$ and its bit neighbors.
Observe that the full code $\ensuremath{\mathcal{C}}$ is simply the intersection of all
the local codes, and the codeword polytope has the \emph{exact}
representation \mbox{$\ensuremath{\operatorname{conv}}(\ensuremath{\mathcal{C}}) =\ensuremath{\operatorname{conv}}(
\bigcap_{\ensuremath{a}=1}^{m}\ensuremath{\mathcal{C}}_\ensuremath{a})$.}  The first-order LP decoder
simply ignores interactions between the various local codes, and
performs the optimization over the relaxed polytope given by
\mbox{$\ensuremath{\mathcal{P}} \ensuremath{: \, = } \bigcap_{\ensuremath{a}=1}^{m}
\ensuremath{\operatorname{conv}}(\ensuremath{\mathcal{C}}_\ensuremath{a})$.}  Note that $\ensuremath{\mathcal{P}}$ is a convex set
that contains the codeword polytope $\ensuremath{\operatorname{conv}}(\ensuremath{\mathcal{C}})$, but also
includes additional vertices with fractional coordinates (called
\emph{pseudocodewords} in the coding literature).  It can be
shown~\cite{WaiJaaWil05b} that if the LDPC graph had no cycles and
hence were tree-structured, this relaxation would be exact;
consequently, this relaxation can be thought of as tree-based.

In sharp contrast to the codeword polytope for a general factor graph
with cycles, the relaxed polytope $\ensuremath{\mathcal{P}}$ for LDPC codes is
always defined by a linear number of constraints.  Consequently, LP
decoding based on solving the relaxed linear program
\begin{equation}
\label{EqnRelaxedLP} \ensuremath{\widehat{y}_{\operatorname{LP}}} = \mathrm{argmin}_{y \in \ensuremath{\mathcal{P}}}
\sum _{i=1}^{n} \gamma_i y_i,
\end{equation}
can solved exactly in polynomial time using standard LP solvers (e.g.,
interior point or simplex methods), or even faster with iterative
methods tailored to the problem
structure~\cite{FelKarWai02,VonKoe_Turbo,Wainwright02aller,WaiJaaWil05b}.


For completeness, we now provide an explicit inequality description of
the relaxed polytope $\ensuremath{\mathcal{P}}$. For every check $\ensuremath{a}$
connected to neighboring variables in the set $N(\ensuremath{a})$ and for all
subsets $S \subseteq N(\ensuremath{a})$, $|S|$ odd, we introduce the
following constraints
\begin{equation}
\label{inequalities} 
\sum_{i \in N(\ensuremath{a})\setminus S} y_i + \sum _{i\in S} (1-y_i) \geq
1.
\end{equation}
Each such inequality corresponds to constraining the $\ell_1$ distance
of the polytope from the sequences not satisfying check
$\ensuremath{a}$---the forbidden sequences---to be at least one. It can be
shown that these \emph{forbidding inequalities} do not exclude valid
codewords from the relaxed polytope.  We also need to add a set of $2
n$ \emph{box inequalities}---namely, $0 \leq y_i \leq 1$---in order to
ensure that we remain inside the unit hypercube. The set of forbidding
inequalities along with the $[0,1]$-box inequalities define the
relaxed polytope.

Note that, given a check $a$ of degree $\ensuremath{d_c}$, there are
$2^{\ensuremath{d_c}-1}$ local forbidden sequences, i.e. sequences of bits in the
check neighborhood $N(a)$ that do not satisfy the check $a$.
Consequently, for a constant check degree code, the total number of
local forbidden sequences is $2^{\ensuremath{d_c} -1 }m$, so that number of
forbidding inequalities scales linearly in the block length
$n$. Fortunately, in the case of low-density parity-check codes, the
degree $\ensuremath{d_c}$ is usually either a fixed constant (for regular
constructions) or small with high probability (for irregular
constructions) so that the number of local forbidden sequences remains
small. Overall, in the cases of practical interest, the relaxed
polytope can be characterized by a linear number of inequalities in
the way that we have described.  (We refer the interested reader to
\cite{Feldman05, FCasc07} for alternative descriptions more
suitable for the case of large $\ensuremath{d_c}$.)

\section{Main result and proof outline}
\label{SecMain}

In this section, we state our main result characterizing the
performance of LP decoding for a random ensemble of LDPC codes, and
provide an outline of the main steps.  Section~\ref{SecProof}
completes the technical details of the proof.


\myparagraph{Random code ensemble} We consider the random ensemble of
codes constructed according to the following procedure.  Given a code
rate $\ensuremath{R} \in (0,1)$, form a bipartite factor graph $\Graph =
(\ensuremath{V}, \ensuremath{C})$ with a set of $\ensuremath{n} = |\ensuremath{V}|$ variable
nodes, and $\ensuremath{m} = |\ensuremath{C}| =
\lfloor(1-\ensuremath{R})\ensuremath{n}\rfloor$ check nodes as follows: (i) Fix a
variable degree $\ensuremath{d_v} \in \mathbb{N}$; and (ii) For each variable
$\jbit \in \ensuremath{V}$, choose a random subset $\ensuremath{N}(\jbit)$ of size
$\ensuremath{d_v}$ from $\ensuremath{C}$, and connect variable $\jbit$ to each check
in $\ensuremath{N}(\jbit)$.  For obvious reasons, we refer to the resulting
ensemble as the \emph{bit-degree-regular} random ensemble, and use
$\code(\ensuremath{d_v})$ to denote a randomly-chosen LDPC code from this
ensemble.

The analysis of this paper focuses primarily on the binary symmetric
channel (BSC), in which each bit of the transmitted codeword is
flipped independently with some probability $\alpha$.  By
concentration of measure for the binomial distribution, it is
equivalent (at least asymptotically) to assume that a constant
fraction $\alpha \ensuremath{n}$ of bits are flipped by the channel.  Let
$\ensuremath{\mathbb{P}}$ denote the joint distribution, over both the space of
bit-degree-regular random codes, and the space of $\alpha \ensuremath{n}$ bit
flips.  With the goal of obtaining upper bounds on the LP error
probability $\ensuremath{\mathbb{P}}[\mbox{LP fails}]$, our analysis is based on the
\emph{expansion} of the factor graph of the code. Specifically, the
factor graph of a code with blocklength $n$ is a $(\ensuremath{\mu},
\ensuremath{p})$-\emph{expander} if all sets $S$ of \emph{variable nodes}, of
size $|S|\leq \ensuremath{\mu} n$, are connected to at least $\ensuremath{p} |S|$
checks.\footnote{Throughout this paper, we work with codes with simple
parity check constraints (LDPC codes) which are different from the
generalized expander codes \cite{ZipSpi96},\cite{lpcap_soda}
that can have large linear subcodes as constraints.}


\myparagraph{Statement of main result} Our main result is a novel
bound on the probability of error for LP decoding, applicable for
finite block length $n$ and the bit-degree-regular LDPC ensemble.  The
main idea is to show that, under certain expansion properties of the
code, LP decoding will succeed in recovering the correct codeword with
high probability.  We note that a random graph will have the required
expansion properties with high probability.

In particular, we show that for the joint distribution over random
expander bit-degree-regular codes and $\lceil\alpha_{c} \ensuremath{n}\rceil$
(or less) bit flips by the channel, there exists a constant
$\alpha_{c}$, depending on the expansion properties of the ensemble,
such that LP decoding succeeds with high probability. More formally,
\begin{theos}
\label{ThmMain}
For every bit-degree-regular LDPC code ensemble with parameters
$\ensuremath{R},\ensuremath{d_v},n$, we specify quadruples $(\alpha_c, c,
\ensuremath{\mu},\ensuremath{p})$ for which the LP decoder succeeds with high
probability over the space of $(\ensuremath{\mu},\ensuremath{p})$-expander
bit-regular random codes and at most $\lceil\alpha_c \ensuremath{n}\rceil$
bit flips.  The probability of failure decreases exponentially in
$c$---namely
\begin{equation}
\label{eq:hehe}
\ensuremath{\mathbb{P}}\left[~\mbox{\em LP success} \, \mid \, \code(\ensuremath{d_v}) \,
\mbox{\emph{is an $(\ensuremath{\mu},\ensuremath{p})$ expander}} \right] \geq
1-e^{-c\ensuremath{n}}. \qquad
\end{equation}
\end{theos}
\noindent 
We note that any factor graph sampled from the bit-regular ensemble
will be an expander with high probability.  In general, the fraction
of correctable errors $\alpha_c$ guaranteed by Theorem~\ref{ThmMain}
is a function of the code ensemble, specified by the code rate
$\ensuremath{R}$, the bit degree $\ensuremath{d_v}$, its expansion parameters
$\ensuremath{\mu}$ and $\ensuremath{p}$ and the error exponent $c$.  For any code
rate, the maximum fraction of correctable errors $\alpha_c$ achieved
by our analysis is provably larger than that of the best previously
known result~\cite{Feldman05b} for LP decoding, which guaranteed
correction of a fraction $\frac{ 3 \ensuremath{p} -2}{2\ensuremath{p} -1} \, \ensuremath{\mu}$
of errors.  As a particular illustration of the stated
Theorem~\ref{ThmMain}, we have the following guarantee for rate $R =
\frac{1}{2}$ codes:
\begin{cors}
\label{cor:nonExpanderCase}
For code rate $\ensuremath{R}=\frac{1}{2}$, bit degree $\ensuremath{d_v}=8$ and error
fraction $\alpha \in (0,0.002)$, the LP decoder succeeds with
probability $1-o(1)$ over the space of bit-degree-regular random codes
of degree $\ensuremath{d_v}$ and $\lceil\alpha \ensuremath{n}\rceil$ bit flips.
\end{cors}
More generally, for any code rate $\ensuremath{R}$, our analysis in
Section~\ref{SecProof} (see discussion following Lemma~\ref{lemma:A3})
specifies conditions for the bit flipping probability $\alpha_c$ and
the expansion parameters $\ensuremath{\mu}$ and $\ensuremath{p}$ so that the
condition~\eqref{eq:hehe} is satisfied with a suitable choice of error
exponent $c$.


\subsection{Outline of main steps}

We now describe the main steps involved in the proof of
Theorem~\ref{ThmMain}.

\subsubsection{Hyperflow witness}
As in previous work~\cite{Feldman05b}, we prove that the LP decoder
succeeds by constructing a \emph{dual witness}: a dual feasible vector
with zero dual cost, which guarantees that the transmitted codeword is
optimal for the primal linear program.  Using the symmetry of the
relaxed polytope, it can be shown~\cite{Feldman05} that the failure or
success of LP decoding depends only on the subset of bits flipped by
the channel and \emph{not} on the transmitted codeword.  Consequently,
we may assume without loss of generality that the all zero codeword
was transmitted. Moreover, note that, for the binary symmetric channel
(BSC) with flip probability $\epsilon$, the log-likelihood of
each received bit is either
$\log{\left(\frac{1-\epsilon}{\epsilon}\right)}$ or
$-\log{\left(\frac{1-\epsilon}{\epsilon}\right)}$. Since the optimum
of the primal is not affected by rescaling, we may assume without loss
of generality that all $\gamma_i$ are either $1$ or $-1$. Then, every
flipped bit $i$ will be assigned $\gamma_i = -1$, whereas every
unflipped bit $\gamma_i=1$. Under these assumptions, Feldman et
al. \cite{Feldman05b} demonstrated that a dual witness can be
graphically interpreted as a set of weights on the edges of the factor
graph of the code:
\begin{lems}[Dual witness~\cite{Feldman05b}]
\label{LemDualWit}
Suppose that all bits in the set $\ensuremath{F}$ are flipped by the
channel, whereas all bit in the complementary set
$\ensuremath{\DirtySet^c}:=V\setminus \ensuremath{F}$ are left unchanged. Set
$\gamma_i=-1$, for all $i \in \ensuremath{F}$, and $\gamma_i=1$, for all
$i\in \ensuremath{\DirtySet^c}$. Then linear programming (LP) decoding succeeds
for this error pattern if there exist weights $\tau_{\ensuremath{i}} \newcommand{\jbit}{\ensuremath{j}\ensuremath{a}}$
for all checks $\ensuremath{a} \in \ensuremath{C}$ and distinct adjacent bits $\ensuremath{i}} \newcommand{\jbit}{\ensuremath{j} \in
N(\ensuremath{a})$ such that the following conditions hold:
\begin{subequations}
\label{EqnDualWit}
\begin{eqnarray}
\label{EqnDualWitA}~~~~~~~\tau_{\ensuremath{i}} \newcommand{\jbit}{\ensuremath{j}\ensuremath{a}} +
\tau_{\jbit\ensuremath{a}} & \geq & 0 \qquad \mbox{$\forall$ checks $\ensuremath{a}
\in \ensuremath{C}$, and}  \\
& & \mbox{$\forall $ adjacent bits $\ensuremath{i}} \newcommand{\jbit}{\ensuremath{j},\jbit \in N(\ensuremath{a})$.}
\nonumber \\
\label{EqnDualWitB} \sum_{\ensuremath{a} \in N(\ensuremath{i}} \newcommand{\jbit}{\ensuremath{j})}
\tau_{\ensuremath{i}} \newcommand{\jbit}{\ensuremath{j}\ensuremath{a}} & < & \gamma_\ensuremath{i}} \newcommand{\jbit}{\ensuremath{j} \qquad \mbox{$\forall \; i \in
V$}.
\end{eqnarray}
\end{subequations}
\label{lemma:feldman et al characterization of LP success} \end{lems}

We next introduce a sufficient condition for the success of LP
decoding, one which is equivalent but arguably more intuitive than the
dual witness definition:
\begin{defns}
\label{DefnHyper}
A {\em hyperflow for $\gamma$} is a set of edge weights $\tau_{ij}$
that satisfy condition~\eqref{EqnDualWitB} and moreover, have the
following additional property: for every check $j \in C$, there exists
a $\mathcal{P} _j \geq 0$ such that for exactly one variable $i \in N(j)$,
$\tau_{ij}= -\mathcal{P}_j$ and for all the other $i' \in N(j) \setminus
\{i\}$, $\tau_{i'j}=\mathcal{P}_j$.
\end{defns}

 The flow interpretation is that each check corresponds to a hyperedge
connecting its adjacent variables; the function of any check is to
replicate the flow incoming from one variable towards all its other
adjacent variables.  With this set-up, condition~\eqref{EqnDualWitB}
corresponds to the requirement that all the variables $i$ with
$\gamma_i < 0$ need to get rid of at least $-\gamma_i$ units of
``poison'', whereas each variable $i$ with $\gamma_i>0$ can absorb at
most $\gamma_i$ units of ``poison''.  Figure~\ref{Hyperflow}
illustrates a valid hyperflow for a simple code.
\begin{figure}[h]
\begin{center}
\widgraph{0.45\textwidth}{hyperflow.eps}
\vspace{-1cm}
 \caption{Example of a valid hyperflow: bits $x_1$ and $x_4$ have been
 flipped in a binary symmetric channel, and hence each are
 contaminated with one unit of poison.  Each of the unflipped bits can
 absorb up to one unit of poison, whereas the checks act as
 hyper-edges and replicate any incoming flow in all directions other
 than the incoming one. The valid hyperflow shown in this figure
 certifies that LP decoding can correct these two flipped bits. }
 \label{Hyperflow}
\end{center}
\end{figure}

We claim that the existence of a valid hyperflow is equivalent to
the dual witness:
\begin{props}
\label{PropHyperflow}
There exists a weight assignment $\tau_{ij}$ satisfying the conditions
of Lemma \ref{lemma:feldman et al characterization of LP success} if
and only if there exists a hyperflow $\tau'_{ij}$ for $\gamma$.
\end{props}
\noindent See Appendix~\ref{AppPropHyperflow} for a proof of this
claim.

\subsubsection{Hyperflow from $(\ensuremath{p}, \ensuremath{q})$ matching}

Let $N(\ensuremath{F})$ denote the subset of checks that are adjacent to
the set $\ensuremath{F}$ of flipped bits.  One way to construct a hyperflow
for $\gamma$ is to match each bit $i$ in the set $F$ of flipped bits,
with some number of checks, say $\ensuremath{p} \le \ensuremath{d_v}$ checks, to which it
has the exclusive privilege to push flow, suppose in a uniform
fashion. This follows because in a matching each check is used at most once. Let us refer to the checks that are actually used in such a
matching as \emph{dirty}, and to all the checks in $\ensuremath{N}(\ensuremath{F})$
as \emph{potentially dirty}. The challenge is that there might be
unflipped variables that are adjacent to a large number of dirty checks, and
hence fail to satisfy condition~\eqref{EqnDualWitB}; i.e. they receive
more flow than they can absorb. Thus, the goal is to construct the
matching of the flipped bits in a careful way so that no unflipped bit
has too many dirty neighbors. The $\delta$-matching witness, used by
Feldman et al.~\cite{Feldman05b}, avoids this difficulty by matching
\emph{all} of the bits adjacent to potentially dirty checks with
$\delta = \ensuremath{p}$ checks each. Our approach circumvents this difficulty
using a more refined combinatorial object that we call a $(\ensuremath{p},
\ensuremath{q})$-matching. For each bit $\jbit \in \ensuremath{\DirtySet^c}$, let
\mbox{$\ensuremath{Z}_\jbit \ensuremath{: \, = } |\ensuremath{N}(\jbit) \cap
\ensuremath{N}(\ensuremath{F})|$} be the number of its edges adjacent to checks in
$\ensuremath{N}(\ensuremath{F})$.
\begin{defns}
\label{def:pqmatching} Given non-negative integers $\ensuremath{p}$ and
$\ensuremath{q}$, a $(\ensuremath{p}, \ensuremath{q})-$\emph{matching} is defined by the
following conditions:
\begin{enumerate}[(a)]
\item each bit $\ensuremath{i}} \newcommand{\jbit}{\ensuremath{j} \in \ensuremath{F}$ must be matched with $\ensuremath{p}$
distinct checks, and
\item each bit $\jbit \in \ensuremath{\DirtySet^c}$ must be matched with
\begin{eqnarray}
\ensuremath{X}_\jbit & \ensuremath{: \, = } & \max\{\ensuremath{q} - \ensuremath{d_v} + \ensuremath{Z}_\jbit, 0
\}
\end{eqnarray}
distinct checks from the set $\ensuremath{N}(\ensuremath{F})$.
\end{enumerate}
\end{defns}
In all theoretical analysis in this paper, it is technically
convenient to consider only pairs $(\ensuremath{p}, \ensuremath{q})$ such that
\begin{equation*}
\ensuremath{p} \, \geq \, \ensuremath{q}, \quad 2\ensuremath{p} + \ensuremath{q} > 2 \ensuremath{d_v}, \quad \mbox{and}
 \quad \ensuremath{d_v} \geq \ensuremath{p} + 2.
\end{equation*}
(The lone exception is Figure~\ref{FigGenMatch}, which is shown only
for illustrative purposes.)

We refer to the number of checks with which each variable node needs
to be matched as its \emph{request number}. In this language, all
flipped bits have $\ensuremath{p}$ requests while each unflipped bit $\jbit$ has
a variable number of requests $\ensuremath{X}_\jbit$ which depends on how
many of its edges land on checks which have flipped neighbors.
\begin{figure}[t]
\begin{center}
\psfrag{#S#}{$\ensuremath{F}$} \psfrag{#nS#}{$N(\ensuremath{F})$}
\psfrag{#Scomp#}{$\ensuremath{\DirtySet^c}$}
\widgraph{.42\textwidth}{genmatching.eps}
\caption{Illustration of a $(\ensuremath{p}, \ensuremath{q})$-generalized matching with
\mbox{$\ensuremath{p} = 2$,} $\ensuremath{q} = 3$ and $\ensuremath{d_v} = 4$.  The first three bits
are flipped, and form the poisoned set $S$; each flipped bit must be
matched with $\ensuremath{p} = 2$ checks from its neighborhood (edges drawn with
arrows). The bit node labeled $2$ lies in $S^c$: it connects to $Z_2 =
|N(2) \cap N(\ensuremath{F})| = 3$ checks within the set $N(\ensuremath{F})$,
and so must be matched with $\ensuremath{X}_2 = \ensuremath{q} - (\ensuremath{d_v} - 3) = 2$
checks from $N(\ensuremath{F})$ (two incoming arrows).  By construction,
bit $2$ then has $2+1 = \ensuremath{q}$ checks that are not contaminated.
Similarly, bit $1$ connects $Z_1 = 2$ checks from $N(\ensuremath{F})$, and
so must be matched with $\ensuremath{X}_1 = \ensuremath{q} - (\ensuremath{d_v}-2) = 1$ check from
$N(\ensuremath{F})$ (incoming arrow).  It has a total of $1 + 2 = \ensuremath{q}$
checks that are not contaminated.}
\label{FigGenMatch}
\end{center}
\end{figure}
The following lemma justifies why requests are selected in this way and illustrates 
the key property of the $(\ensuremath{p},
\ensuremath{q})$-matching:
\begin{lems}
\label{pqmatch} A $(\ensuremath{p},\ensuremath{q})$-matching guarantees that
all the flipped bits are matched with $\ensuremath{p}$ checks, and all
the non-flipped bits have $\ensuremath{q}$ or more non-dirty check
neighbors.
\end{lems}
\noindent This fact follows by observing that any unflipped bit
$\jbit$ with $\ensuremath{Z}_\jbit$ edges in $\ensuremath{N}(\ensuremath{F})$ has
$\ensuremath{d_v}-\ensuremath{Z}_\jbit$ clean neighboring checks, and requests
$\ensuremath{q} -(\ensuremath{d_v}-\ensuremath{Z}_\jbit)$ extra checks from the potentially
dirty ones.

Figure~\ref{FigGenMatch} illustrates a generalized matching for a
degree $\ensuremath{d_v} = 4$ factor graph, and $(\ensuremath{p}, \ensuremath{q}) = (2,3)$.  Note
that the bit node labeled $2$ has $Z_2 = 3$ neighbors in the
potentially dirty set $N(\ensuremath{F})$, and so it makes $\ensuremath{X}_2 = 3
- (4-3) = 2$ requests for matching.  This ensures that it is connected
to $2+1 = \ensuremath{q}$ checks that are not dirty.  A similar argument
applies to bit $1$, with $Z_1 = 2$ and $\ensuremath{X}_1 = 1$.

We next claim that a $(\ensuremath{p}, \ensuremath{q})$-matching is a certificate of LP
decoding success:
\begin{lems}
\label{LemStrongerWitness} For any integers $\ensuremath{p}$ and $\ensuremath{q}$ such that $2
\ensuremath{p} + \ensuremath{q} > 2 \ensuremath{d_v}$, a $(\ensuremath{p}, \ensuremath{q})$-matching can be used to
generate a set of weights $\tau_{\ensuremath{i}} \newcommand{\jbit}{\ensuremath{j}\ensuremath{a}}$ which constitute a
hyperflow for $\gamma$ and, hence, satisfy the dual witness
conditions~\eqref{EqnDualWit}.
\end{lems}
\begin{proof} Each flipped bit is matched with $\ensuremath{p}$ checks: suppose it sends
$\chi$ units of poison to each of these checks.  In the worst case,
the remaining $\ensuremath{d_v}-\ensuremath{p}$ edges are connected to checks to which
other flipped bits are sending poison.  Therefore, each flipped bit
(in the worst case) can purge itself of $\ensuremath{p} \chi -(\ensuremath{d_v}-\ensuremath{p}) \chi$
units of its own poison, so that we require that \mbox{$\ensuremath{p} \chi
-(\ensuremath{d_v}-\ensuremath{p}) \chi > 1$.}

By Lemma~\ref{pqmatch}, each unflipped bits has at least $\ensuremath{q}$
checks that do not send any poison.  In the worst case, then, an
unflipped bit can receive $(\ensuremath{d_v}-\ensuremath{q}) \chi$ units of poison, which
we require to be less than $1$.  Overall, a valid routing parameter
$\chi$ will exist if $\frac{1}{2\ensuremath{p} -\ensuremath{d_v}} < \frac{1}{\ensuremath{d_v}-\ensuremath{q}}$,
or equivalently, if $\ensuremath{p} + \ensuremath{q} > 2 \ensuremath{d_v}$ as claimed.
\end{proof}

In fact, it can verified that our combinatorial witness for LP
decoding success is easier to satisfy than the condition used by
Feldman et al.~\cite{Feldman05b}. Our use of this improved witness,
along with our focus on the probabilistic setting, are the two
ingredients that allow us to establish a much larger fraction
$\alpha_c$ of correctable errors.


\subsubsection{From expansion to matching via Hall's theorem}

The remainder (and bulk) of the analysis involves establishing that,
with high probability over the selection of random expander
bit-degree-regular codes and random subsets of $\lceil\alpha
\ensuremath{n}\rceil$ flipped bits, a $(\ensuremath{p}, \ensuremath{q})$-matching exists, for
suitable values $\ensuremath{q} \le \ensuremath{p}$ to be specified later.  It is well
known~\cite{BM01,ZipSpi96} that random regular bipartite graphs will
have good expansion, with high probability:
\begin{lems}[Good expansion]
\label{lemma:expander}
For any fixed code rate $\ensuremath{R} \in (0,1)$, degree $\ensuremath{d_v}$ and
$\ensuremath{p} \le \ensuremath{d_v}-2$, there exist constants $\ensuremath{\mu}, c>0$ so that
a code $\code(\ensuremath{d_v})$ from the bit-degree-regular ensemble of degree
$\ensuremath{d_v}$ is a $(\ensuremath{\mu} \ensuremath{n}, \ensuremath{p})$ expander with probability
at least $1-O(1/n)$.  \end{lems}

Therefore, conditioned that the event that the random graph is an
expander, the next step is to analyze the existence of a $(\ensuremath{p},
\ensuremath{q})$- matching.  We use Hall's theorem~\cite{vanLint}, which in our
context, states that a matching exists if and only if \emph{every
subset of the variable nodes} have (jointly) enough neighbors in
$\ensuremath{N}(\ensuremath{F})$ to cover the sum of their requests.


Given our random graph and channel models, an equivalent description
of the neighborhood choices for each variable $\jbit \in
\ensuremath{\DirtySet^c}$ is as follows.  Each node $\jbit \in \ensuremath{\DirtySet^c}$
picks a random number $\ensuremath{Z}_\jbit \in \{0,1,\ldots,\ensuremath{d_v}\}$
according to the binomial distribution
$\ensuremath{\operatorname{Bin}}(\ensuremath{d_v},\frac{|\ensuremath{N}(\ensuremath{F})|}{m})$, and picks a subset of
$\ensuremath{N}(\ensuremath{F})$ of size $\ensuremath{Z}_\jbit$. This subset corresponds
to the intersection of its check neighborhood $\ensuremath{N}(j)$ with the
check neighborhood $\ensuremath{N}(\ensuremath{F})$ of the flipped bits.  The
remaining $\ensuremath{d_v} - \ensuremath{Z}_j$ edges from bit $\jbit$ connect to checks
outside $\ensuremath{N}(\ensuremath{F})$.  With this set-up, we now define the a
``bad event'' ${\ensuremath {\mathcal E}}$, defined by the existence of a pair
$(S_1,S_2) \in 2^{\ensuremath{F}}\times 2^{\ensuremath{\DirtySet^c}}$ of sets that
\emph{contracts}, meaning that it has more requests than neighbors, so
that
\begin{multline}
\label{EqnContract}
\left|\ensuremath{N}(S_1) \cup \left[\ensuremath{N}(S_2) \cap \ensuremath{N}(\ensuremath{F})
\right] \right| < \\ \ensuremath{p} |S_1| + \sum_{\jbit \in S_2}
\max\{0,\ensuremath{q}-(\ensuremath{d_v} - \ensuremath{Z}_\jbit) \} \Big\}.
\end{multline}

Notice that only the neighbors in $\ensuremath{N}(\ensuremath{F})$ are counted,
since a $(\ensuremath{p}, \ensuremath{q})$-matching involves only checks in
$\ensuremath{N}(\ensuremath{F})$. By Lemma~\ref{LemStrongerWitness}, the event
${\ensuremath {\mathcal E}}$ must occur whenever LP decoding fails so that we have the
inequality $\ensuremath{\mathbb{P}}[\mbox{LP decoding fails}] \leq \ensuremath{\mathbb{P}}[{\ensuremath {\mathcal E}}]$.
Defining the event 
\begin{eqnarray}
\ensuremath{\mathcal{G}}(\ensuremath{d_v}, \ensuremath{\mu}, \ensuremath{p}) & \ensuremath{: \, = } & \left \{
  \code(\ensuremath{d_v}) \; \mbox{is a $(\ensuremath{\mu} n,\ensuremath{p})$ expander} \right
  \},
\end{eqnarray}
we make use of the following conditional form of this inequality:
\begin{eqnarray*}
\ensuremath{\mathbb{P}}\left[~\mbox{LP decoding fails} \, \mid \, \ensuremath{\mathcal{G}} \right]
  & \leq & \ensuremath{\mathbb{P}}\left[ {\ensuremath {\mathcal E}} \; \mid \; \ensuremath{\mathcal{G}} \right].
\end{eqnarray*}
It is useful to partition the space $2^\ensuremath{F} \times
2^{\ensuremath{\DirtySet^c}}$ into three subsets controlled by the parameters
$\epsilon_2,\ensuremath{\mu} >0$. Parameter $\epsilon_2 > 0$ is a small constant
to be specified later in the proof and $\ensuremath{\mu}$ is the expansion
coefficient.
The three subsets of interest are given by
\begin{subequations}
\begin{eqnarray}
A_1 & \ensuremath{: \, = } & \{(S_1,S_2) \mid (S_1,S_2)\in A, \; |S_1|+|S_2| <
\ensuremath{\mu} n\} \qquad \\
A_2 & \ensuremath{: \, = } & \{(S_1,S_2) \mid (S_1,S_2)\in A-A_1, \; |S_1| \ge \epsilon_2
n\}, \qquad \\
A_3 & \ensuremath{: \, = } & A \backslash \left(A_1 \cup A_2 \right).
\end{eqnarray}
\end{subequations}
This partition, as illustrated in Figure~\ref{Fig:partitionOfSpace},
\begin{figure}[t]
\begin{center}
\widgraph{0.45\textwidth}{fig_part_space.eps}
\caption{Partitioning the space $2^\ensuremath{F} \times
2^{\ensuremath{\DirtySet^c}}$.}
\label{Fig:partitionOfSpace}
\end{center}
\end{figure}
decomposes ${\ensuremath {\mathcal E}}$ into sub-events
\begin{eqnarray*}
{\ensuremath {\mathcal E}}(A_i)& \ensuremath{: \, = } & \left \{ \exists (S_1,S_2)\in A_i \; \Biggr |
\; \mbox{equation~\eqref{EqnContract} holds} \right \}
\end{eqnarray*}
for $i=1,2,3$.  Then, via a series of union bounds, we have the
following upper bound on the probability of failure
\begin{eqnarray*}
\ensuremath{\mathbb{P}}[~\mbox{LP fails} \mid \ensuremath{\mathcal{G}}] & \leq & \ensuremath{\mathbb{P}}[~{\ensuremath {\mathcal E}}
\; \mid \ensuremath{\mathcal{G}}] \\
& \leq & \sum_{i=1}^3 \ensuremath{\mathbb{P}}[~{\ensuremath {\mathcal E}}(A_i) \, \mid \, \ensuremath{\mathcal{G}}].
\end{eqnarray*}
However, all subsets of variable nodes of size at most $\ensuremath{\mu}
n$ in a $(\ensuremath{\mu} n,\ensuremath{p})$ expander have a $\ensuremath{p}$-matching and,
because $\ensuremath{q} \le \ensuremath{p}$, it follows that
\begin{eqnarray}
\label{eq:kostassmallsets}
\ensuremath{\mathbb{P}}[~{\ensuremath {\mathcal E}}(A_1) \, \mid \, \ensuremath{\mathcal{G}}] & = & 0.
\end{eqnarray}
Consequently, we only have to deal with the remaining two terms of the
summation for $i = 2$ and $i=3$.  Before proceeding, an important
side-remark here is that equation~\eqref{eq:kostassmallsets} by itself
implies that the LP decoder can correct a constant fraction of errors;
indeed, it is precisely this observation that was exploited by Feldman
et al.~\cite{Feldman05b}.  However, our ultimate goal in this paper
is to establish higher fractions of correctable errors, so we need to
continue our analysis further.

For $i=2,3$, we have
\begin{eqnarray}
\ensuremath{\mathbb{P}} \left[{\ensuremath {\mathcal E}}(A_i) \; \mid \; \ensuremath{\mathcal{G}} \right] & = &
\frac{\ensuremath{\mathbb{P}}[~{\ensuremath {\mathcal E}}(A_i)\wedge
\ensuremath{\mathcal{G}}]}{\ensuremath{\mathbb{P}}[\ensuremath{\mathcal{G}}]} \nonumber \\
& \leq & \frac{\ensuremath{\mathbb{P}}[~{\ensuremath {\mathcal E}}(A_i)~]}{\ensuremath{\mathbb{P}}[\ensuremath{\mathcal{G}}]}
\nonumber\\
& \leq & 2 \ensuremath{\mathbb{P}}[~{\ensuremath {\mathcal E}}(A_i)~],
\end{eqnarray}
where the last inequality follows from Lemma~\ref{lemma:expander}.
Overall, putting everything together, we conclude that
\begin{eqnarray}
\label{middle bound}
\ensuremath{\mathbb{P}}\left[~\mbox{LP fails} \: \mid \; \ensuremath{\mathcal{G}} \right] & \leq &
2\sum_{i=2}^3{\ensuremath{\mathbb{P}}[~{\ensuremath {\mathcal E}}(A_i)]}.
\end{eqnarray}

The remainder of the proof consists of careful analysis of these two
error terms.  It turns out to be convenient to use an alternative
probabilistic model in the analysis.  In particular, observe that
there is an inconvenient asymmetry in the definition of our
generalized matching: the bits of set $\ensuremath{\DirtySet^c}$ need to be
matched with checks from the neighborhood of the flipped bits
$\ensuremath{F}$, and not from the whole set of checks from which they
select their neighbors. This correlation between $\ensuremath{N}(F)$ and the
number of requests from set $\ensuremath{\DirtySet^c}$ creates severe
complications in the analysis. Indeed, any attempt to use Hall's
condition through union bounds seems to require independence among
different edges; moreover, crude upper-bounds on the number of
requests from set $\ensuremath{\DirtySet^c}$ seem inadequate to decorrelate the
requests of $\ensuremath{\DirtySet^c}$ from the size of $\ensuremath{N}(F)$.  For this
reason, we use an alternative probabilistic model, as described in
Section~\ref{section:newProbabilisticModel}.



\section{Proof of Theorem~\ref{ThmMain}}
\label{SecProof}

We now turn to the remaining (somewhat more technical) steps involved
in the proof of Theorem~\ref{ThmMain}.

\subsection{Simplifying the probability model}
\label{section:newProbabilisticModel}

In order to decouple the distribution of the requests of
$\ensuremath{\DirtySet^c}$ from the size of $\ensuremath{N}(\ensuremath{F})$, observe that the
number of requests $\ensuremath{X}_j$ from each bit $j$ in $\ensuremath{\DirtySet^c}$
grows linearly with the number of edges that this bit has in
$\ensuremath{N}(\ensuremath{F})$. 
Notice that the checks are selected with replacement and the degree of a variable 
can be strictly smaller than $d_v$, although this will not be an issue asymptotically. 
This observation combined with a coupling
argument shows that, if $x, x' \in \{0,\ldots,d_v\}^{|\ensuremath{\DirtySet^c}|}$
are two vectors of requests from the bits in $\ensuremath{\DirtySet^c}$, where $x
\leq x'$ elementwise, then the probability that a
$(\ensuremath{p},\ensuremath{q})$-matching exists is larger conditioned on $x$ than on
$x'$.


This observation suggests the following alternative
experiment:
\begin{itemize}
\item A node $\jbit \in \ensuremath{\DirtySet^c}$ first picks a random number
\mbox{$\ensuremath{Z}_\jbit \in \{0,1,\ldots,\ensuremath{d_v}\}$} according to the
modified binomial distribution
$\ensuremath{\operatorname{Bin}}\left(\ensuremath{d_v},\frac{\ensuremath{d_v}\lceil\alpha
\ensuremath{n}\rceil}{m}\right)$.
\item Node $\jbit$ then chooses $\ensuremath{Z}_\jbit$ checks from
$\ensuremath{N}(\ensuremath{F})$ with replacement.
\end{itemize}
This procedure is repeated independently for each $\jbit \in
\ensuremath{\DirtySet^c}$.  Since $|\ensuremath{N}(\ensuremath{F})| \le \ensuremath{d_v}\lceil\alpha
\ensuremath{n}\rceil$, the bits of set $\ensuremath{\DirtySet^c}$ will tend to have more
edges in $\ensuremath{N}(\ensuremath{F})$ and, therefore, more requests in this new
experiment than in the original one (as suggested by the natural
coupling between the two processes). Moreover, since checks are now
chosen with replacement, for each bit $\jbit \in \ensuremath{\DirtySet^c}$, the
size of the intersection $\ensuremath{N}(j) \cap \ensuremath{N}(\ensuremath{F})$ is less
than or equal to $\ensuremath{Z}_\jbit$, since the same check might be chosen
more than once. Intuitively, the existence of matchings is less likely
in the new experiment than in the original one; this claim follows
rigorously by combining these observations with the coupling argument
used in the previous paragraph. The benefit of switching from the
original experiment to this new experiment is in allowing us to
\emph{decouple} the process of deciding the number of requests made by
each bit in $\ensuremath{\DirtySet^c}$ from the cardinality of the random
variable $\ensuremath{N}(\ensuremath{F})$.

Let us use $\ensuremath{\mathbb{Q}}$ to denote the probability distribution over random
graphs in this new model. Setting $\ensuremath{\DirtySet^c}(\ensuremath{q}) = \left \{ j
\in \ensuremath{\DirtySet^c} \; \mid \; \ensuremath{q} > \ensuremath{d_v} - \ensuremath{Z}_\jbit \right
\}$, we can define the alternative ``bad event'' ${\ensuremath {\mathcal B}}$, meaning
that there exist $S_1 \subseteq \ensuremath{F}$, and $S_2 \subseteq
\ensuremath{\DirtySet^c}(\ensuremath{q})$ such that
\begin{equation}
\label{EqnContract2}
\left|\ensuremath{N}(S_1) \cup \left[\ensuremath{N}(S_2) \cap \ensuremath{N}(\ensuremath{F})
\right] \right| \leq \ensuremath{p} |S_1| + \sum_{\jbit \in S_2}
\left[\ensuremath{q}-(\ensuremath{d_v} - \ensuremath{Z}_\jbit) \right]. \qquad
\end{equation}
In addition, we define the corresponding sub-events ${\ensuremath {\mathcal B}}(A_i)$
for $i=1,2,3$. As argued above, it must hold that
\begin{equation*}
\ensuremath{\mathbb{P}}[{\ensuremath {\mathcal E}}(A_i)] \leq \ensuremath{\mathbb{Q}}[{\ensuremath {\mathcal B}}(A_i)], \quad \mbox{for
all} \quad i = 1, 2, 3,
\end{equation*}
and, therefore, as inequality~\eqref{middle bound} suggests, in order
to upper bound the probability of LP decoding failure, it suffices to
obtain upper bounds on the probabilities $\ensuremath{\mathbb{Q}}[{\ensuremath {\mathcal B}}(A_i)]$ for
$i=2,3$. 

For future use, we define for fixed subsets $S_1 \subseteq \ensuremath{F}$
and $S_2 \subseteq \ensuremath{\DirtySet^c}(\ensuremath{q})$, the event
${\ensuremath {\mathcal B}}(S_1,S_2)$ that equation~\eqref{EqnContract2} holds for
$S_1$ and $S_2$.  We now proceed, in a series of steps, to obtain
suitable upper bounds on the probabilities $\ensuremath{\mathbb{Q}}[{\ensuremath {\mathcal B}}(A_i)]$
and, hence, on the probability of LP decoding failure.


\subsection{Conditioning on requests from $\ensuremath{\DirtySet^c}$} 
\label{section:mainPROOF}

For each $i \in \{1, \ldots, \ensuremath{q} \}$, we define the random variable
\begin{equation}
\ensuremath{Y}_i \ensuremath{: \, = } \biggr | \{ \jbit \in \ensuremath{\DirtySet^c} \; \mid \;
\ensuremath{Z}_\jbit = \ensuremath{d_v} - (\ensuremath{q} - i) \} \biggr |,
\end{equation}
corresponding to the number of bits in $\ensuremath{\DirtySet^c}$ with $\ensuremath{d_v} -
(\ensuremath{q} -i) $ edges that lie inside the ``contaminated'' neighborhood
$\ensuremath{N}(\ensuremath{F})$ and, hence, with $i$ requests each.  Note that
each $\ensuremath{Y}_i$ is binomial with parameters $\lfloor(1-\alpha)
\ensuremath{n}\rfloor$ and
\begin{equation}
\ensuremath{b}_i \ensuremath{: \, = } {\ensuremath{d_v} \choose {\ensuremath{d_v}-\ensuremath{q} +i}}
\left(\frac{\lceil\alpha\ensuremath{n}\rceil\ensuremath{d_v}}{m} \right)^{\ensuremath{d_v} - \ensuremath{q}
+i} \left(1-\frac{\lceil\alpha\ensuremath{n}\rceil\ensuremath{d_v}}{m} \right)^{\ensuremath{q} -
i}.
\end{equation}
Since \mbox{$\ensuremath{\mathbb{E}}[\ensuremath{Y}_i] = \ensuremath{b}_i \lfloor(1-\alpha) \,
\ensuremath{n}\rfloor$}, applying Hoeffding's inequality~\cite{Hoeffding63}
yields the sharp concentration
\begin{equation*}
\ensuremath{\mathbb{Q}}\left[|\ensuremath{Y}_i - \ensuremath{b}_i\lfloor(1-\alpha) \, \ensuremath{n}\rfloor|
\geq \epsone \ensuremath{n} \right] \; \leq \; 2 \exp \left ( -2 \epsone^2
\ensuremath{n} \right)
\end{equation*}
for any $\epsone > 0$.  Hence, if we define the event
\begin{equation}
\ReqTail(\epsone) \ensuremath{: \, = } \bigcap_{i=1}^{\ensuremath{q}} \left \{ |\ensuremath{Y}_i -
\ensuremath{b}_i\lfloor(1-\alpha) \ensuremath{n}\rfloor| \leq \epsone \ensuremath{n}
\right \},
\end{equation}
 then a simple union bound yields that
\begin{equation*}
\ensuremath{\mathbb{Q}}[\ReqTail(\epsone)] \geq 1 - 2 \ensuremath{q} \; \exp \left ( -2
\epsone^2 \ensuremath{n} \right),
\end{equation*}
 so that it suffices to bound the conditional probabilities
$\ensuremath{\mathbb{Q}}[{\ensuremath {\mathcal B}}(A_i) \, \mid \, \ReqTail(\epsone)]$, $i=2,3$.  Note
that conditioned on the event $\ReqTail(\epsone)$, we are guaranteed
that
\begin{eqnarray}
\label{EqnDefnRboubar}
\frac{\ensuremath{Y}_i}{\ensuremath{n}} & \leq & \ensuremath{b}_i (1-\alpha) + \epsone \; =
\,: \; \ensuremath{\widebar{y}^{\operatorname{up}}}_i.
\end{eqnarray}


We now turn to bounding the probability of the bad event ${\ensuremath {\mathcal B}}$.
Since, by symmetry, the probability of the event ${\ensuremath {\mathcal B}}(S_1,S_2)$
is the same for different sets $S_1$ of the same size, a union bound
gives $\ensuremath{\mathbb{Q}}[{\ensuremath {\mathcal B}}(A_2)\, \mid \, \ReqTail(\epsone)] \leq
\sum_{s_1= \lceil\epstwo \ensuremath{n}\rceil}^{\lceil\alpha \,
\ensuremath{n}\rceil} \ensuremath{D}(s_1)$, where 
\begin{multline*}
\ensuremath{D}(s_1) \ensuremath{: \, = } { \lceil\alpha \ensuremath{n} \rceil \choose s_1} \times \\
\ensuremath{\mathbb{Q}} \Big[ \exists~S_2 \subseteq \ensuremath{\DirtySet^c}(\ensuremath{q}) \; \mbox{with}
\; (S_1,S_2) \in A_2 \mbox{ s.t.  } {\ensuremath {\mathcal B}}(S_1, S_2) \; \Big |
\ReqTail(\epsone)\Big],
\end{multline*}
with $S_1$ is any fixed set of size $s_1$.

Before bounding these terms, we first partition the values of $s_1$
into two sets $\{\lceil\epsilon_2\ensuremath{n}\rceil,\ldots,\lceil\ensuremath{\sbar_{\operatorname{crit}}}
\ensuremath{n}\rceil \}$ and $\{\lceil\ensuremath{\sbar_{\operatorname{crit}}} \ensuremath{n}\rceil+1,\ldots,
\lceil\alpha \ensuremath{n}\rceil \}$ for some value of $\ensuremath{\sbar_{\operatorname{crit}}}$ to be
specified formally in Lemma \ref{LemRegOne}.  To give some intuition,
in the conditional space $\ReqTail(\epsone)$, the total number of
matching-requests from the bits of set $\ensuremath{\DirtySet^c}$ is at most
\begin{equation}
V \ensuremath{: \, = } \ensuremath{n} \sum_{i=1}^\ensuremath{q} i \ensuremath{\widebar{y}^{\operatorname{up}}}_i.
\end{equation}
Therefore, if $\ensuremath{\mathbb{Q}}[{\ensuremath {\mathcal B}}(A_2)\, \mid \,\ReqTail(\epsone)]$ is
indeed small, we would expect that, if the set $S_1$ is large enough
(say $|S_1| \approx |\ensuremath{F}|$), then with high probability, the
size of its image $\ensuremath{N}(S_1)$ should be large enough not only to
cover its own requests but also $V$ additional
requests---viz. $|\ensuremath{N}(S_1)| \ge p|S_1| + V$.  If this condition
holds, then there cannot exist any set $S_2$ such that the event
${\ensuremath {\mathcal B}}(S_1,S_2)$ occurs.  We formalize this intuition in the
following result, proved in Appendix~\ref{AppRegOne}:
\begin{lems}[Upper Regime]
\label{LemRegOne}
Define \mbox{$\ensuremath{\widebar{v}} \ensuremath{: \, = } \sum_{i=1}^\ensuremath{q} i \;
\ensuremath{\widebar{y}^{\operatorname{up}}}_i$,} and the function 
\begin{equation*}
f(s) \ensuremath{: \, = } \alpha H\left(\frac{s}{\alpha}\right) +
(1-\ensuremath{R})H\left(\frac{\ensuremath{p} s + \ensuremath{\widebar{v}}}{(1-\ensuremath{R})}\right) +
\ensuremath{d_v} s\log_2 \left(\frac{\ensuremath{p} s +
\ensuremath{\widebar{v}}}{(1-\ensuremath{R})}\right),
\end{equation*}
where $H(\cdot)$ is the binary entropy function, and set
\begin{eqnarray}
\label{EqnDefnSbaronecrit}
\ensuremath{\sbar_{\operatorname{crit}}} \ensuremath{: \, = } \min \left \{ \alpha, \; \inf \left\{ s \in [0,
\alpha] \; \mid \; f(s') < 0, \forall s' \in [s,\alpha] \right\}
\right \}.
\end{eqnarray}
Then for all $s_1 \in \{\lceil\ensuremath{\sbar_{\operatorname{crit}}} \ensuremath{n}\rceil+1,\ldots,
\lceil\alpha n\rceil\}$, the quantity $\ensuremath{D}(s_1)$ decays
exponentially fast in $n$.
\end{lems}
\noindent It remains to bound $\ensuremath{D}(s_1)$ for $s_1 \in \ensuremath{{L_I}}$, where
\begin{equation*}
\ensuremath{{L_I}} \ensuremath{: \, = } \{\lceil\epstwo \ensuremath{n}\rceil,\ldots, \lceil\ensuremath{\sbar_{\operatorname{crit}}}
\ensuremath{n}\rceil\}. 
\end{equation*}
For a randomly chosen set $S_1$, define the event 
\begin{equation}
\mathcal{F}(s_1, \ensuremath{\gamma_1}) \ensuremath{: \, = } \{|S_1| = s_1, \quad |\ensuremath{N}(S_1)| = \ensuremath{\gamma_1}
\}.
\end{equation}
By conditioning, we have the decomposition $\ensuremath{D}(s_1) =
\sum_{\ensuremath{\gamma_1}=1}^{\ensuremath{d_v} s_1} \ensuremath{E}(\ensuremath{\gamma_1}, s_1)$, where
\begin{multline*}
\ensuremath{E}(\ensuremath{\gamma_1}, s_1) \ensuremath{: \, = } {\lceil\alpha \ensuremath{n}\rceil \choose s_1 }
\; \ensuremath{\mathbb{Q}}'[\mathcal{F}(s_1, \ensuremath{\gamma_1})] \times \\ \ensuremath{\mathbb{Q}}'\left[ \exists \;
S_2 \mbox{ with } (S_1,S_2)\in A_2 \mbox{ s.t. } {\ensuremath {\mathcal B}}(S_1, S_2)
\; \Big| \mathcal{F}(s_1, \ensuremath{\gamma_1}) \right].
\end{multline*}
Here $\ensuremath{\mathbb{Q}}'$ denotes the conditional probability distribution of
$\ensuremath{\mathbb{Q}}$ conditioned on the event $\ReqTail(\epsone)$.


The following lemma allows us to restrict our attention to
linearly-sized check neighborhoods $\ensuremath{N}(S_1)$ in analyzing the
individual terms $\ensuremath{E}(\ensuremath{\gamma_1}, s_1)$ of the summation; the proof is
provided in Appendix~\ref{appendix:LinearSizeNeighborhood}.
\begin{lems}[$\ensuremath{\widebar{\gamma}_1}$ small]
\label{LemLinSizeCheckNei} 
Define the critical point $\ensuremath{\bar{\gamma}_{\operatorname{crit}}}(\ensuremath{\widebar{s}}} \long\def\comment#1{_1)$
\begin{equation}
\label{EqnSup}
\sup \left \{ \ensuremath{\widebar{\gamma}_1} \in (0,\ensuremath{d_v} \ensuremath{\widebar{s}}} \long\def\comment#1{_1]
\; \mid \; 2 + \ensuremath{d_v} \ensuremath{\widebar{s}}} \long\def\comment#1{_1 \log_2
\left(\frac{\ensuremath{\widebar{\gamma}_1}}{(1-\ensuremath{R})} \right) < 0 \right\}.
\end{equation}
Then, for set sizes $s_1 \geq \lceil\epstwo \ensuremath{n}\rceil $ and
neighborhood sizes \mbox{$\ensuremath{\gamma_1} \leq \ensuremath{\bar{\gamma}_{\operatorname{crit}}}(\epstwo) \ensuremath{n}$,} the
quantity $\ensuremath{E}(\ensuremath{\gamma_1}, s_1)$ decays exponentially fast in
$\ensuremath{n}$.
\end{lems} 
\noindent Note that the supremum~\eqref{EqnSup} is always finite.
This lemma essentially says that if $s_1$ has linear size, its neighborhood
$\ensuremath{\gamma_1}$ must also have linear size. 
\vspace*{.2in}


 To summarize our progress thus far, we first argued that in order to
bound the probability $\ensuremath{\mathbb{Q}}[{\ensuremath {\mathcal B}}(A_2)\, \mid \,
\ReqTail(\epsone)]$, it suffices to bound the quantities
$\ensuremath{D}(s_1)$, for $s_1 \in \{\lceil\epstwo
\ensuremath{n}\rceil,\ldots,\lceil\alpha \ensuremath{n}\rceil\}$. Next we
partitioned the range of $s_1$ into two sets: the lower set $\ensuremath{{L_I}} =
\{\lceil\epsilon_2 n\rceil,\ldots,\lceil\ensuremath{\sbar_{\operatorname{crit}}} n\rceil \}$, and the
upper set $\ensuremath{{U_I}} \ensuremath{: \, = } \{\lceil\ensuremath{\sbar_{\operatorname{crit}}} \ensuremath{n}\rceil+1,\ldots,
\lceil\alpha \ensuremath{n}\rceil \}$.  The upper set has the property that
for all sets $S_1 \subseteq \ensuremath{F}$ of size $|S_1| \in \ensuremath{{U_I}}$, with
high probability the neighborhood $\ensuremath{N}(S_1)$ is big enough to
accommodate not only the matching requests from set $S_1$, but also
all possible matching-requests from any set $S_2 \subseteq
\ensuremath{\DirtySet^c}$. Having established this property of large $S_1$ sets,
it remains to focus on small $S_1$. In this regime, the neighborhood
$\ensuremath{N}(S_1)$ on its own is no longer sufficient to cover the joint
set of requests from $S_1$ \emph{and} from any possible set $S_2
\subseteq \ensuremath{\DirtySet^c}$.  Consequently, one has to consider for every
choice $(S_1, S_2) \in A_2$, whether the joint neighborhood
$\ensuremath{N}(S_1) \cup (\ensuremath{N}(S_2) \cap \ensuremath{N}(\ensuremath{F}))$ is large
enough to cover the matching requests from $S_1$ and $S_2$.

At this point, one might imagine that a rough concentration argument
applied to the sizes of $\ensuremath{N}(S_1)$ and $\ensuremath{N}(S_2)\cap
\ensuremath{N}(\ensuremath{F})\setminus \ensuremath{N}(S_1)$ would suffice to complete the
proof.  Unfortunately, any concentration result must be sufficiently
strong to dominate the factor $\lceil\alpha n\rceil \choose s_1$ that
leads the expression $\ensuremath{D}(s_1)$.  Consequently, we study the exact
distribution of the size of $\ensuremath{N}(S_1)$, and bound the quantities
$\ensuremath{E}(\ensuremath{\gamma_1}, s_1)$ for $s_1 \in \ensuremath{{L_I}}$ and $\ensuremath{\gamma_1} \in
\{1,\ldots,\ensuremath{d_v} s_1\}$. Of course, since $s_1$ is linear in size, the
bulk of the probability mass is concentrated on linear values for
$\ensuremath{\gamma_1}$. Therefore, by Lemma \ref{LemLinSizeCheckNei}, we need only
bound $\ensuremath{E}(\ensuremath{\gamma_1}, s_1)$ for $s_1 \in \ensuremath{{L_I}}$ and $\ensuremath{\gamma_1} \ge
\ensuremath{\bar{\gamma}_{\operatorname{crit}}}(\epstwo)\ensuremath{n}$.  We complete these steps in the following
subsection.




\subsection{Completing the bound}

Let us fix sizes $s_1 \in \ensuremath{{L_I}}$ and $\ensuremath{\gamma_1} \ge \ensuremath{\bar{\gamma}_{\operatorname{crit}}}(\epstwo)
\ensuremath{n}$. For a set $S_1$ of size $s_1$ with neighborhood
$\ensuremath{N}(S_1)$ of size $\ensuremath{\gamma_1}$, define its {\em residual neighborhood}
to be the set $\ensuremath{N}(\ensuremath{F}) \ensuremath{\backslash} \ensuremath{N}(S_1)$ and use
\mbox{$\ensuremath{\gamma_2} \ensuremath{: \, = } |\ensuremath{N}(\ensuremath{F}) \ensuremath{\backslash} \ensuremath{N}(S_1)|$} to denote
its size.  Moreover, define the vector of requests\footnote{Recall
that we have conditioned on the event $\ReqTail(\epsone)$, so that the
number of bits in $\ensuremath{\DirtySet^c}$ with $i$ matching requests is
concentrated, for every $i\in\{1,\ldots,q\}$.}  $\ensuremath{\vec{\myreq}} \in
\otimes_{i=1}^{q}\{0,..,\lceil\ensuremath{\widebar{y}^{\operatorname{up}}}_i\ensuremath{n}\rceil\}$, let us denote
by $\ensuremath{\beta}(s_1,\ensuremath{\gamma_1},\ensuremath{y})$ the number of checks missing from the
neighborhood of $S_1$ to cover the total number of requests from $S_1$
and a set $S_2$ with configuration of requests
$\ensuremath{y}$. Also, let $\ensuremath{\nu}(\ensuremath{y})$ be the number of edges from $S_2$ to $\ensuremath{N}(\ensuremath{F})$. More
precisely, the quantities $\ensuremath{\beta}(s_1,\ensuremath{\gamma_1},\ensuremath{y})$ and
$\ensuremath{\nu}(\ensuremath{y})$ are given by the formulae
\begin{subequations}
\begin{eqnarray}
\label{EqnDefnGamThree}
\ensuremath{\beta}(s_1,\ensuremath{\gamma_1},\ensuremath{\vec{\myreq}}) & \ensuremath{: \, = } & \ensuremath{p} s_1 - \ensuremath{\gamma_1} +
\sum_{i=1}^{\ensuremath{q}}{i \ensuremath{y}_i} \\
\ensuremath{\nu}(\ensuremath{\vec{\myreq}}) & \ensuremath{: \, = } & \sum_{i=1}^{\ensuremath{q}}{(\ensuremath{d_v}-\ensuremath{q}+i) \,
\ensuremath{y}_i}.
\end{eqnarray}
\end{subequations}
Note that for $s_1 \in \ensuremath{{L_I}}$ and $\ensuremath{\gamma_1} \ge \ensuremath{\bar{\gamma}_{\operatorname{crit}}}(\epstwo)
\ensuremath{n}$, the quantity $\ensuremath{\beta}$ also grows linearly in $\ensuremath{n}$;
as usual, we use $\ensuremath{\widebar{\gamthree}}$ to denote the rescaled quantity
$\ensuremath{\beta}/\ensuremath{n}$.  Also recall the definition of $\ensuremath{\widebar{y}^{\operatorname{up}}}_i$ from
equation~\eqref{EqnDefnRboubar}.  

Letting $\ensuremath{\widebar{\myreq}} \ensuremath{: \, = } (\ensuremath{\widebar{\myreq}}_1, \ldots, \ensuremath{\widebar{\myreq}}_\ensuremath{q})$ be a
vector of request fractions in $[0,1]^\ensuremath{q}$, we define
\begin{eqnarray*}
\Gfunc(\ensuremath{\widebar{s}}} \long\def\comment#1{_1, \ensuremath{\widebar{\gamma}_1}, \ensuremath{\widebar{\gamma}_2}, \ensuremath{\vec{\myreq}}) & \ensuremath{: \, = } & \sum_{k=1}^4
\min \{ 0, \Gfunc_k(\ensuremath{\widebar{s}}} \long\def\comment#1{_1, \ensuremath{\widebar{\gamma}_1}, \ensuremath{\widebar{\gamma}_2}, \ensuremath{\widebar{\myreq}}) \}
\end{eqnarray*}
where 
\begin{subequations}
\begin{eqnarray*}
\Gfunc_1 & \ensuremath{: \, = } & \alpha H\left(\frac{\ensuremath{\widebar{s}}} \long\def\comment#1{_1}{\alpha}\right)
\sum_{i=1}^{\ensuremath{q}} \ensuremath{\widebar{y}^{\operatorname{up}}}_i
H\left(\frac{\ensuremath{\widebar{\myreq}}_i}{\ensuremath{\widebar{y}^{\operatorname{up}}}_i}\right)\\
\Gfunc_2 & \ensuremath{: \, = } & (1-\ensuremath{R})
H\left(\frac{\ensuremath{\widebar{\gamma}_1}}{(1-\ensuremath{R})}\right) + \ensuremath{d_v} \ensuremath{\widebar{s}}} \long\def\comment#1{_1
\log_2\left(\frac{\ensuremath{\widebar{\gamma}_1}}{(1-\ensuremath{R})}\right) \\
\Gfunc_3 & \ensuremath{: \, = } & ((1-\ensuremath{R})-\ensuremath{\widebar{\gamma}_1})
H\left(\frac{\ensuremath{\widebar{\gamma}_2}}{((1-\ensuremath{R})-\ensuremath{\widebar{\gamma}_1})} \right) \\
& & \qquad + \ensuremath{d_v} (\alpha-\ensuremath{\widebar{s}}} \long\def\comment#1{_1) \log_2 \left(\frac{\ensuremath{\widebar{\gamma}_1} +
\ensuremath{\widebar{\gamma}_2} }{(1-\ensuremath{R})}\right) \\
\Gfunc_4 & \ensuremath{: \, = } & \ensuremath{\widebar{\gamma}_2}
H\left(\frac{\min\{\ensuremath{\widebar{\gamma}_2},\ensuremath{\widebar{\gamthree}}(\ensuremath{\widebar{s}}} \long\def\comment#1{_1,\ensuremath{\widebar{\gamma}_1},\ensuremath{\widebar{\myreq}})\}}{\ensuremath{\widebar{\gamma}_2}}\right)
\\
& & \qquad
+ \ensuremath{\nu}(\ensuremath{\widebar{\myreq}}) \log_2\left(\frac{\ensuremath{\widebar{\gamma}_1}+\min\{\ensuremath{\widebar{\gamma}_2},
\ensuremath{\widebar{\gamthree}}(\ensuremath{\widebar{s}}} \long\def\comment#1{_1,\ensuremath{\widebar{\gamma}_1},\ensuremath{\widebar{\myreq}})\} }{\ensuremath{\widebar{\gamma}_1}+\ensuremath{\widebar{\gamma}_2}}\right)
\end{eqnarray*}
\end{subequations}
With these definitions, we have the following result:
\begin{lems}[Exponential upper bound]
\label{LemExpUpper} Suppose that the following inequalities hold:
\begin{subequations}
\label{EqnThreeInequal}
\begin{eqnarray}
\ensuremath{\sbar_{\operatorname{crit}}} < \frac{\alpha}{2}, &&\\
\alpha \ensuremath{d_v} < \frac{(1-\ensuremath{R})-\ensuremath{d_v} \ensuremath{\sbar_{\operatorname{crit}}}}{2}, \quad
\mbox{and} && \\
\alpha H\left(\frac{\ensuremath{\sbar_{\operatorname{crit}}}}{\alpha} \right) \ensuremath{d_v}
(\alpha-\ensuremath{\sbar_{\operatorname{crit}}}) \log_2 \left(\frac{\ensuremath{d_v}
\ensuremath{\sbar_{\operatorname{crit}}}}{(1-\ensuremath{R})} \right) \; < \; 0. &&
\end{eqnarray}
\end{subequations}
Then for some $c > 0$, we have the exponential upper bound
\begin{eqnarray*}
\ensuremath{\mathbb{Q}}[{\ensuremath {\mathcal B}}(A_2)\, \mid \, \ReqTail(\epsone)] & \leq &
 2^{\ensuremath{n} \KeyFunc(\alpha)} + \exp(-c n)
\end{eqnarray*}
where the function in the exponent is given by
\begin{eqnarray}
\label{EqnDefnKeyFunc}
F(\alpha) & \ensuremath{: \, = } & \sup_{\ensuremath{\widebar{s}}} \long\def\comment#1{_1,\ensuremath{\widebar{\gamma}_1},\ensuremath{\widebar{\gamma}_2},\{\ensuremath{\widebar{\myreq}}_i\}}{
\Gfunc(\ensuremath{\widebar{s}}} \long\def\comment#1{_1, \ensuremath{\widebar{\gamma}_1}, \ensuremath{\widebar{\gamma}_2},
\ensuremath{\widebar{\myreq}}_1,\ldots,\ensuremath{\widebar{\myreq}}_{\ensuremath{q}})},
\end{eqnarray}
with the maximization over 
\begin{eqnarray*}
\ensuremath{\widebar{s}}} \long\def\comment#1{_1 & \in & [0,\ensuremath{\sbar_{\operatorname{crit}}}] \\ 
\ensuremath{\widebar{\gamma}_1} & \in & [0,\ensuremath{d_v} \ensuremath{\widebar{s}}} \long\def\comment#1{_1] \\
\ensuremath{\widebar{\gamma}_2} & \in &  [0, \; \ensuremath{d_v}\, (\alpha- \ensuremath{\widebar{s}}} \long\def\comment#1{_1) ], \quad \mbox{and} \\
\widebar{y}_i & \in & [\ensuremath{\widebar{y}^{\operatorname{up}}}_i/2,\ensuremath{\widebar{y}^{\operatorname{up}}}_i].
\end{eqnarray*}
\end{lems}
\noindent See Appendix~\ref{AppExpUpper} for a proof of this
lemma.

\vspace*{.1in}

It remains to upper bound the probability of the bad-event
${\ensuremath {\mathcal B}}(A_3)$ which is equivalent to the existence of a pair of
contracting sets $(S_1,S_2)$, where the size of set $S_1\subseteq
\ensuremath{F}$ is at most $\epstwo n$ and the size of set $S_2 \subseteq
\ensuremath{\DirtySet^c}$ is at least $(\ensuremath{\mu} - \epstwo) n$. Note that we
haven't yet specified the constant $\epstwo$. The following lemma
establishes that there exists a value of $\epstwo$ so that
$\ensuremath{\mathbb{Q}}[{\ensuremath {\mathcal B}}(A_3)\, \mid \, \ReqTail(\epsone)]$ is bounded by an
exponentially decreasing function in $n$ provided that the function
$F(\alpha)$ from equation~\eqref{EqnDefnKeyFunc} is negative.  The
proof of this final lemma is provided in Appendix
\ref{appendix:proofForA3}.

\begin{lems}
\label{lemma:A3} If $F(\alpha)<0$, then there exists $\epstwo$
so that the probability $\ensuremath{\mathbb{Q}}[{\ensuremath {\mathcal B}}(A_3)\, \mid \,
\ReqTail(\epsone)]$ is decreasing exponentially in $n$.
\end{lems}

We may now complete the proof of Theorem~\ref{ThmMain}.  For a given
rate $\ensuremath{R}$, fix the bit degree $\ensuremath{d_v}$ and the matching
parameters $(\ensuremath{p}, \ensuremath{q})$ such that
\begin{equation}
\label{EqnCondOne}
2\ensuremath{p}+\ensuremath{q}>2\ensuremath{d_v}, \quad \ensuremath{p}\ge\ensuremath{q}, \quad \mbox{and} \quad
\ensuremath{d_v}-\ensuremath{p} \ge 2,
\end{equation}
 and recall the definition~\eqref{EqnDefnSbaronecrit} of
$\ensuremath{\sbar_{\operatorname{crit}}}$.  Suppose that the three
inequalities~\eqref{EqnThreeInequal} hold, and that the function
$F$ defined in equation~\eqref{EqnDefnKeyFunc} satisfies
\begin{eqnarray}
\label{EqnCondThree}
F(\alpha) & < & 0
\end{eqnarray}
Then the probability
\begin{equation*}
\ensuremath{\mathbb{P}}[~\mbox{LP decoding fails} \, \mid \, \code(\ensuremath{d_v})\mbox{ is a
$(\ensuremath{\mu} n,\ensuremath{p})$ expander}~]
\end{equation*}
decays exponentially in $n$, where $\ensuremath{\mathbb{P}}$ is the uniform distribution
over the set of bit-degree-regular codes of degree $\ensuremath{d_v}$ and
selections of $\lceil \alpha n\rceil$ bit flips.

In particular, these explicit conditions allow us to investigate
fractions of correctable errors on specific code ensembles.  As a
concrete example, for code rate $\ensuremath{R} = 1/2$, if we choose
variable degrees $\ensuremath{d_v} = 8$ and generalized matching parameters
$(\ensuremath{p}, \ensuremath{q}) = (6,5)$, one can numerically verify that the
conditions~\eqref{EqnCondOne}, \eqref{EqnCondThree}
and~\eqref{EqnThreeInequal} are satisfied for all $\alpha \leq
\ensuremath{\alpha_{\operatorname{crit}}} = 0.002$. Therefore, for that rate, we establish a fraction
of correctable errors which is more than ten times higher than the
previously known worst-case results, as claimed.

\section{Conclusion}
\label{SecDiscussion}

The main contribution of this paper is to perform probabilistic
analysis of linear programming (LP) decoding of low-density
parity-check (LDPC) codes in the finite-length regime. Specifically,
we showed that for a random LDPC code ensemble, the linear programming
decoder of Feldman et al. succeeds (with high probability) in
correcting a constant fraction of errors that surpasses all prior
non-asymptotic results for LDPC codes.  For a rate $0.5$ code, it
exceeds the best previous finite-length result on LP decoding by a
factor greater than ten.  Despite these substantial improvement, it should be noted that our analysis still yields very
conservative results, roughly a factor of 50 lower than the typical
empirical performance of these codes, as well as the associated
asymptotic thresholds.

Perhaps more important than specific numerical improvements over past
results are the technical innovations that underlie our analysis: a
direct treatment of the probabilistic nature of bit-flipping channels
(as opposed to adversarial analysis in previous work), and a novel
combinatorial characterization of LP decoding, based on the
notion of a poison hyperflow witness.  This hyperflow perspective
illustrates that the factor graph defining a good code should have
good flow properties, in the sense that no matter which subset of bits
are flipped, the poison associated with errors can be diffused and
routed to the unflipped bits. 
For more general MBIOS channels, the amount of poison corresponds exactly to the negative log-likelihood that the channel is assigning to each bit, and the same characterization of LP decoding holds.  

 This intuition suggests that the
property of supporting sufficient hyperflow could provide a useful
design principle in the finite-length setting, for example small sets of variables which contract (are jointly adjacent to few checks) will cause pseudocodewords of small pseuodoweight. 

  There are a number of
ways in which specific technical aspects of the current analysis can
likely be sharpened, which await further work.  In addition, it
remains to further explore the consequences of our analysis technique
for other channels and code ensembles, beyond the particular LDPC
ensemble and binary symmetric channel considered here.

\subsection*{Acknowledgements}

The authors would like the thank the anonymous referees for their 
numerous and detailed suggestions that
helped to improve this paper.  This work was
presented in part in January 2007 at the Symposium on Discrete
Algorithms (SODA), New Orleans, LA.


\bibliographystyle{plain}


2008-03-10T05:13:09.000
0702
cs/0702014
en
https://arxiv.org/abs/cs/0702014
We initiate the probabilistic analysis of linear programming (LP) decoding of low-density parity-check (LDPC) codes. Specifically, we show that for a random LDPC code ensemble, the linear programming decoder of Feldman et al. succeeds in correcting a constant fraction of errors with high probability. The fraction of correctable errors guaranteed by our analysis surpasses previous non-asymptotic results for LDPC codes, and in particular exceeds the best previous finite-length result on LP decoding by a factor greater than ten. This improvement stems in part from our analysis of probabilistic bit-flipping channels, as opposed to adversarial channels. At the core of our analysis is a novel combinatorial characterization of LP decoding success, based on the notion of a generalized matching. An interesting by-product of our analysis is to establish the existence of ``probabilistic expansion'' in random bipartite graphs, in which one requires only that almost every (as opposed to every) set of a certain size expands, for sets much larger than in the classical worst-case setting.
Information Theory (cs.IT); Discrete Mathematics (cs.DM)
Probabilistic Analysis of Linear Programming Decoding
Qwen/Qwen-72B
1. YES
2. YES