Numerous recent approaches attempt to remove image blur due to camera shake, either with one or multiple input images, by explicitly solving an inverse and inherently ill-posed deconvolution problem. If the photographer takes a burst of images, a modality available in virtually all modern digital cameras, we show that it is possible to combine them to get a clean sharp version. This is done without explicitly solving any blur estimation and subsequent inverse problem. The proposed algorithm is strikingly simple: it performs a weighted average in the Fourier domain, with weights depending on the Fourier spectrum magnitude. The method can be seen as a generalization of the align and average procedure, with a weighted average, motivated by hand-shake physiology and theoretically supported, taking place in the Fourier domain. The method’s rationale is that camera shake has a random nature, and therefore, each image in the burst is generally blurred differently. Experiments with real camera data, and extensive comparisons, show that the proposed Fourier burst accumulation algorithm achieves state-of-the-art results an order of magnitude faster, with simplicity for on-board implementation on camera phones. Finally, we also present experiments in real high dynamic range (HDR) scenes, showing how the method can be straightforwardly extended to HDR photography.
@cite_0: This paper addresses the problem of obtaining a quality photograph of a painting by multi-image fusion methods. The problem is particularly challenging because of the uncontrolled illumination conditions and of the destructive reflection speckle present in most photographs of paintings. A fully automatic image processing chain is described that, starting from several bursts of a painting taken under different angles, permits one to obtain the best possible result by eliminating highlights and motion blur by robust statistics, reducing noise by fusion, and compensating optical distortion in the registration process. This image fusion method is applicable to photographs of a painting taken with a hand-held camera without any particular setup. It works under bad lighting conditions and eliminates motion blur, even when the painting is protected by a glass screen creating structured reflections of the room. The careful discussion of each step of the processing chain also permits one to review and discuss the ...
@cite_5: A dynamic procedure for selective information fusion from multiple image frames is derived from robust error estimation theory. The fusion rate is driven by the anisotropic gain function, defined to be the difference between the Gaussian smoothed-edge maps of a given input frame and of an evolving synthetic output frame. The gain function achieves both selection and rapid fusion of relatively sharper features from each input frame compared to the synthetic frame. Effective applications are demonstrated for image sharpening in imaging through atmospheric turbulence, for multispectral fusion of the RGB spectral components of a scene, for removal of blurred visual obstructions from in front of a distant focused scene, and for high-resolution two-dimensional display of three-dimensional objects in microscopy.
@cite_4: An automated video enhancement technique capable of image fusion from a stream of randomly-distorted images of a still scene is presented in this paper. The technique is based on the "lucky-region" fusion (LRF) approach and aims to improve locally the image quality according to the following steps: (1) for each image of the video stream an image quality map (IQM) which characterizes locally the image quality is computed, (2) each IQM is compared to that of the current fused image leading to the selection of best quality regions (the "lucky-regions"), and (3) the selected regions are merged into the fused video stream. While the LRF approach succeeds in producing images with significantly improved image quality compared to the source images, its performance depends on the imaging conditions and requires adjustment of its fusion parameter - the fusion kernel size - in order to adapt to an evolving environment (e.g. a turbulent atmosphere). Parameter selection was so far performed manually using a trial-and-error approach which causes the technique to be impractical for a real world implementation. The automated LRF technique presented is relaxed from this requirement and selects automatically the fusion parameter based on the analysis of the source images making it more suitable for practical systems. The improved LRF technique is applied to imaging through atmospheric turbulence for various imaging conditions and scenes of interest. In each case automatically-fused video streams demonstrate increases in image quality comparable to that obtained with manual selection of the fusion parameter.
@cite_8: Photographing distant objects is challenging for a number of reasons. Even on a clear day, atmospheric haze often represents the majority of light received by a camera. Unfortunately, dehazing alone cannot create a clean image. The combination of shot noise and quantization noise is exacerbated when the contrast is expanded after haze removal. Dust on the sensor that may be unnoticeable in the original images creates serious artifacts. Multiple images can be averaged to overcome the noise, but the combination of long lenses and small camera motion as well as time varying atmospheric refraction results in large global and local shifts of the images on the sensor. An iconic example of a distant object is Mount Rainier, when viewed from Seattle, which is 90 kilometers away. This paper demonstrates a methodology to pull out a clean image of Mount Rainier from a series of images. Rigid and non-rigid alignment steps brings individual pixels into alignment. A novel local weighted averaging method based on ideas from “lucky imaging” minimizes blur, resampling and alignment errors, as well as effects of sensor dust, to maintain the sharpness of the original pixel grid. Finally, dehazing and contrast expansion results in a sharp clean image.
Others propose to measure the local sharpness from the norm of the gradient or the image Laplacian @cite_5 @cite_4 @cite_8 @cite_0 . Joshi and Cohen @cite_8 engineered a weighting scheme to balance noise reduction and sharpness preservation. The sharpness is measured through the intensity of the image Laplacian. They also proposed a local selectivity weight to reflect the fact that more averaging should be done in smooth regions. Haro and colleagues @cite_0 explored similar ideas to fusion different acquisitions of painting images. The weights for combining the input images rely on a local sharpness measure based on the energy of the image gradient. The main disadvantage of these approaches is that they only rely on sharpness measures (which by the way is not necessarily trivial to estimate) and do not profit the fact that camera shake blur can be in different directions in different frames.