Noisy Parity (LWE) lower bounds/hardness results
The LPN problem is indeed believed to be hard, but like most problems we believe are hard, the main reason for it is that many smart people have tried to find an efficient algorithm and failed.

The best "evidence" for LPN's hardness comes from the high statistical query dimension of the parity problem.  Statistical queries capture most known learning algorithms, except for gaussian elimination (which fails whenever noise is introduced), hashing, and techniques similar to these two.  It is hard to design non statistical-query algorithms, and this is the main bottleneck.  Other evidence of LPN's hardness is its relationship to other hard problems (like LWE, SVP as you've pointed out).

For the SQ-hardness, here is the link to Kearns's ('98) paper. 

For progress on the upper bounds on this problem, there are several results:


Probably the most famous is the Blum-Kalai-Wasserman ('00) result which beats the $2^N$ SQ barrier by a little, giving an algorithm that runs in time $2^{n / \log n}$. (link)
Lyubashevsky ('05) found an algorithm with worse running time $O(2^{n / \log\log n})$ but better sample complexity of $O(n^{1 + \epsilon})$. (link)
In the sparse case, where we know the parity is on $k$ variables, Grigorescu-Reyzin-Vempala ('11) gave a $\approx O(n^{0.5 k})$ algorithm, beating the $O(n^k)$ brute-force.  However, this bound decays toward $O(n^k)$ as the noise rate $\eta$ approaches $1/2$. (link)
Valiant ('12) recently gave a $\approx O(n^{0.8 k})$ algorithm for the sparse case.  This bound is stronger because it does not decay in the exponent with the noise rate (unlike the GRV result). (link)