1. An image processing device comprising: a CPU; and a memory that includes a program, wherein the CPU reads the program from the memory and executes functions as: a weight calculator that determines an area where a feature value of an input image is saved, based on a gradient of a feature value of a pixel of the input image and a direction of the gradient, and calculates a weight for reducing a regularization constraint that is a constraint based on regularization of image processing in the area where the feature value is saved; a regularization term calculator that calculates a regularization constraint of a high resolution image restored based on the input image by using the weight; a reconstruction constraint calculator that calculates a reconstruction constraint that is a constraint based on reconstruction of the high resolution image; and an image restorer that restores the high resolution image from the input image based on the regularization constraint and the reconstruction constraint.