What is the point of this lambda expression?
The “true” lambda calculus is defined by a syntax of terms:
$$
M,N ::= x \;\mid\; \lambda x.M \;\mid\; M\,N
$$
and a reduction rule called beta reduction:
$$
(\lambda x.M)\,N \to M[x \leftarrow N] \tag{$\beta$}
$$
(Plus context rules, plus the rules for alpha equivalence, but I'll omit them in this post.)

A lambda term is an object defined by the syntax above. What makes lambda terms interesting is the way they can be evaluated using the reduction rules.

It's possible, and common, to define extensions of the lambda calculus: languages that have additional syntactic forms and reduction rules. One common, simple extension is to add constants to the syntax of terms, and to add reduction rules called delta rules that allow evaluating these constants.

For example, we can define a new calculus which is the lambda calculus plus elementary arithmetic (LCPEA). If $n$ is a non-negative integer, I'll write $(n)_a$ for a constant in the LCPEA, so the LCPEA has constants $(0)_a$, $(1)_a$, $(42)_a$, etc. Note that what's in the parentheses is an integer, not a string: the constant is the same constant regardless of how the integer is written, so $(0)_a$, $(3-3)_a$, $(1+3-2*2)_a$, etc. are all the same constant.

$$
M_a,N_a ::= x \;\mid\; \lambda x.M_a \;\mid\; M_a\,N_a
        \;\mid\; (n)_a \;\mid\; \mathord{+}_a \;\mid\; \mathord{-}_a \;\mid\; \mathord{*}_a
$$ $$ \begin{align}
(\lambda x.M_a)\,N_a &amp;\to M_a[x \leftarrow N_a] \tag{$\beta$} \\
\mathord{+}_a (n)_a \, (n')_a &amp;\to (n+n')_a \tag{$\delta_+$} \\
\mathord{-}_a (n)_a \, (n')_a &amp;\to (n-n')_a \qquad \text{if \(n \ge n'\)} \tag{$\delta_-$} \\
\mathord{*}_a (n)_a \, (n')_a &amp;\to (n*n')_a \tag{$\delta_*$} \\
\end{align} $$

The delta rules directly translate the arithmetic operators into the LCPEA. So the LCPEA effectively contains elementary arithmetic. Note that these delta rules are necessary if we want the new constants we introduced to do anything useful. It would have been possible to define a different set of delta rules that caused $+_a$ to have nothing to do with addition and so on, but it would have been confusing.

It turns out that it is possible to define pure lambda terms (i.e. terms in the true lambda calculus) that reduce in the same way as corresponding terms in the LCPEA. That is, we can define a lambda term $\langle n\rangle$ for every natural integer $n$, as well as lambda terms $\langle\mathord{+}\rangle$, $\langle\mathord{-}\rangle$ and $\langle\mathord{*}\rangle$ such that
$$ \begin{align}
\langle\mathord{+}\rangle \, \langle n\rangle \, \langle n'\rangle &amp;\leftrightarrow_\beta^* \langle n+n'\rangle \\
\langle\mathord{-}\rangle \, \langle n\rangle \, \langle n'\rangle &amp;\leftrightarrow_\beta^* (n-n')_a \qquad \text{if \(n \ge n'\)} \\
\langle\mathord{*}\rangle \, \langle n\rangle \, \langle n'\rangle &amp;\leftrightarrow_\beta^* (n*n')_a \\
\end{align} $$
The Church numerals are a classical example of a set of such terms; there are others.

There are several reasons why we may want to define something like the LCPEA.


One is that it is closer to how machines behave; this is important when using the lambda calculus to model compilers, but not when doing theory of computation.
Another reason, applicable even when doing theory of computation, is that we often don't care exactly how the integers are encoded. So we might define things like $\langle 3\rangle$ and $\langle\mathord{+}\rangle$ as lambda terms, and then forget their precise definition and just remember some reduction rules for them. So they're still pure lambda terms, but we treat them as if they were part of an extended language.
In a typed lambda calculus, encoding everything as functions is problematic, because we'd like to say that something like $1$ is “inert”, not like a function, and to say that $1 \, 2$ (“1 applied to 2”) is not well-typed and does not reduce to anything. With the pure lambda calculus, $\langle 1\rangle \, \langle 2\rangle$ does beta-reduce to something, but not to something useful. When we extend the language with constants, we don't define any reduction rule that would reduce $(1)_a \, (2)_a$.
There are extensions of the language that cannot be encoded as lambda terms at all. If we're interested in studying these, we have to extend the language.