Gaze correction aims to redirect the person's gaze into the camera by manipulating the eye region, and it can be considered as a specific image resynthesis problem. Gaze correction has a wide range of applications in real life, such as taking a picture with staring at the camera. In this paper, we propose a novel method that is based on the inpainting model to learn from the face image to fill in the missing eye regions with new contents representing corrected eye gaze. Moreover, our model does not require the training dataset labeled with the specific head pose and eye angle information, thus, the training data is easy to collect. To retain the identity information of the eye region in the original input, we propose a self-guided pretrained model to learn the angle-invariance feature. Experiments show our model achieves very compelling gaze-corrected results in the wild dataset which is collected from the website and will be introduced in details. Code is available at this https URL.
@cite_12: In this work, we consider the task of generating highly-realistic images of a given face with a redirected gaze. We treat this problem as a specific instance of conditional image generation and suggest a new deep architecture that can handle this task very well as revealed by numerical comparison with prior art and a user study. Our deep architecture performs coarse-to-fine warping with an additional intensity correction of individual pixels. All these operations are performed in a feed-forward manner, and the parameters associated with different operations are learned jointly in the end-to-end fashion. After learning, the resulting neural network can synthesize images with manipulated gaze, while the redirection angle can be selected arbitrarily from a certain range and provided as an input to the network.
The core idea for most of the method based on machine learning is to use large training set labeled with specific eye angle and head pose information to learn the warping field. The warping field is used to relocate eye pixels in the original image, thus realizing gaze redirection. @cite_12 proposed a novel deep model based on the convolutional network as a predictor, which helps them to achieve a high-quality result. We present a novel method for gaze correction. With an encode-decode architecture as a generator and adversarial process, we can learn from the face image to fill in the missing regions with new contents representing corrected eye gaze. Compared with those learning-based methods, our training dataset is unpaired and collected from the website, where the dataset is not labeled with specific eye angle and head pose information, thus, is easy to collect and use. Furthermore, these models are hard to achieve high-quality results for gaze correction in the wild images.