Mathmatical way to represent an image kernel?
Applying the filter corresponds to two-dimensional convolution:
$$
(f \ast g)(x,y) = \sum_{i=-\infty}^\infty \sum_{j=-\infty}^\infty f(i,j) g(x-i,y-j).
$$
In our case, $f$ is the filter, $g$ is the image, and $f \ast g$ is the filtered image. Since the support of $f$ (the number of non-zero entries) is finite, the sums in question are finite, and the convolution is well-defined. More explicitly, we have $f(0,0) = 5$, $f(-1,0) = f(1,0) = f(0,-1) = f(0,1) = -1$, and $f(i,j) = 0$ otherwise.

Edit 1: Here is another way of viewing it. This time, $f$ is the image and $g$ is the filter, defined by $g(0,0) = 5$, $g(-1,0) = g(1,0) = g(0,-1) = g(0,1) = -1$, and $g(i,j) = 0$. The same formula for filtering, $f \ast g$, works before, since $f \ast g = g \ast f$. In other words, the filtered image $h$ is given by the formula
$$ h(x,y) = \sum_{i=-\infty}^\infty \sum_{j=-\infty}^\infty g(i,j) f(x-i,y-j). $$

Edit 2: We can rewrite the convolution as a product of generating functions. Suppose the original image is given by
$$ F = \sum_x \sum_y f(x,y) X^x Y^y. $$
The filter $G$ in your case is given by
$$ G = 5 - X - X^{-1} - Y - Y^{-1}. $$
Filtering the image corresponds to the multiplication $H = FG$, and the filtered image $h$ can be extracted as
$$ H = \sum_x \sum_y h(x,y) X^x Y^y. $$

Edit 3: Going back to the notation of Edit 1, in terms of the two-dimensional Fourier transform, we have
$$ \hat{h} = \hat{f} \hat{g}, $$
where the multiplication on the right is pointwise. So filtering corresponds to multiplying each Fourier coefficient by some fixed amount, which depends on the filter.

Edit 4: In this particular case, the relation between the original image $f$ and the filtered image $h$ is
$$ h(x,y) = 5f(x,y) - f(x-1,y) - f(x+1,y) - f(x,y-1) - f(x,y+1). $$

Edit 5: The image you posted handles the boundary issue by extending the first and last rows and columns past the boundary. For example, $f(x,-1) = f(x,0)$, $f(-1,y) = f(0,y)$ and $f(-1,-1) = f(0,0)$.