Estimating an Image Filter from Examples
I will talk about the Linear Spatially Invariant (LSI) Model.

Since the Filter (System) is LSI it can be described in the domain of Fourier Transform:

$$ Y \left( \omega \right) = H \left( \omega \right) X \left( \omega \right) $$

Where $ Y \left( \omega \right) $ is the Blurred (Output) Image, $ X \left( \omega \right) $ is the Input Image and $ H \left( \omega \right) $ is the filter model.

Now, assuming the filter doesn't vanish in the Fourier Domain, then:

$$ H \left( \omega \right) = \frac{Y \left( \omega \right)}{X \left( \omega \right)} $$

Now, to practical issues with this model:


Noise.
Vanishing Frequencies.


There are many methods to deal with those issues (Pseudo Inverse, Wiener Filter, Lucy Richardson) all are having different assumptions (White Noise, Gaussian Noise, Linear Model, Poisson Noise, etc...).

Till now I described the classic "Signal Processing" Model.

What you can do is build this model in the Spatial Domain.
The Filtering Operator will be a very large Toeplitz Matrix which you can estimate using LS Model or any solver for any cost function you'd want.

Practically in the world of Image Processing, the quality is in the regularization yo enforce.

P.S.
There is a new community proposal in the SE network:

http://area51.stackexchange.com/proposals/86832

It would be great if you could follow, up vote question with less than 10 points and propose your own questions.

Thank You.