> 
> Currently reading Platt's paper, *Sequential Minimal Optimization: A Fast Algorithm for Training Support Vector Machines* , I got stuck in section 2.3 *Computing the Threshold*:
> 
> 
> 

### SVM notation

---

objective function:
\begin{array}{1}
 \max \_{\alpha }\sum \_{i=1}^{n}\alpha \_{i}-{\frac {1}{2}}\sum \_{i=1}^{n}\sum \_{j=1}^{n}y\_{i}y\_{j}K\_{ij}\alpha \_{i}\alpha \_{j}\\
 0\leqslant \alpha\_i \leqslant C : Lagrange multipliers\\
\sum\_{i=1}^Nyi\alpha\_i=0\\
\end{array}

KKT condition:

\begin{array}{l}
\quad {a\_i} = 0 \quad \Leftrightarrow \quad {y\_i}u\_i \ge 1\\
0 < {a\_i} < C \quad \Leftrightarrow \quad {y\_i}u\_i = 1\\
\quad {a\_i} = C \quad \Leftrightarrow \quad {y\_i}u\_i \le 1
\end{array}

$b$: threshold in SVM model $w^Tx-b$

$u\_i=\sum\_{j=1}^Ny\_j\alpha\_jK\_{ij}-b$: predict value using SVM

$E\_i=u\_i-y\_i$: difference between target and prediction

$K\_{ij}=K(x\_i, x\_j)=K(x\_j,x\_i)$: the kernel matrix

---

### Brief description about SMO

---

According to Platt, SMO optimize two Lagrange multipliers one time, for example:

$y\_1\alpha\_1+y\_2\alpha\_2=-\sum\_{i=3}^Ny\_i\alpha\_i=Const$
[![SMO-SUB](https://i.stack.imgur.com/cu35w.png)](https://i.stack.imgur.com/cu35w.png)

...

Update $\alpha\_i$ 

...

The question
------------

1. if $\alpha\_i$ is not at bound, threshold $b$ can be computed by forcing the output to be $y\_i$: 
$b\_i=E\_i+y\_i(\alpha^{new}\_1-\alpha\_1)K\_{11}+y\_2(\alpha\_2^{new,clipped}-\alpha\_2)K\_{12}+b^{old}$ (eq.1)
2. if both $\alpha\_1$ and $\alpha\_2$ are at bound, then using eq.1 computing $b\_1$ and $b\_2$, all thresholds between $b\_1$ and $b\_2$ are consistent with KKT conditions.

---

I understand case 1 since $0<\alpha\_i<C$,we get $y\_iu\_i=1$, prediction error must be 0, but I failed to understand case 2...