Abstract
We present a novel system to assist users in choosing suitable exposure values for photography on smartphones. The user specifies the desired shape of the histogram of the captured image by adjusting the parameters of a function. The system then uses the smartphone's camera and varying exposure values to calculate a high dynamic range (HDR) histogram of the scene. It contains the entire brightness range from the darkest to the brightest area of the scene. For any exposure value, the HDR histogram can be used to synthesize a conventional histogram of a low dynamic range (LDR) image as if captured by this exposure value. This process is much faster than capturing the image itself. By comparing a number of synthesized LDR histograms to the user's desired histogram shape, the system determines an exposure value that best meets the user's preference.
INTRODUCTION
Even in smartphone photography, users have a desire to take pictures that are visually pleasing. Deciding upon a suitable exposure value for a given scene is one of the aspects that make a good picture. The automatic exposure control in smartphones is often too simple to match the user's preferences. In professional photography, it is common to display brightness histograms to facilitate the process of finding good exposure. In our work, we take a backwards approach: The user specifies the desired shape of the histogram and the system calculates an exposure value that best meets the user's intentions. In order to predict the shape of the histogram for any given exposure value without going through the costly image capturing process, we Permission to make digital or hard copies of part or all of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage, and that copies bear this notice and the full citation on the first page. Copyrights for third-party components of this work must be honored. For all other uses, contact the owner/author(s). Copyright is held by the author/owner(s). MM'13, October 21-25, 2013, Barcelona, Spain. ACM 978-1-4503-2404-5/13/10. http://dx.doi.org/10.1145/2502081.2502252. make use of high dynamic range (HDR) histograms. A conventional low dynamic range (LDR) histogram simulating any exposure value can be directly calculated from the HDR histogram. In the end, the exposure for which the LDR histogram meets the user's preferences best is used to actually take a picture. The entire system is implemented as a camera app for android-based smartphones. A screenshot of it is shown in Figure 1 .
INTRODUCTION
There exist previous works on determining good exposure settings. Many of them focus on finding sequences of shutter values to be used to capture an HDR frame. As opposed to our work where HDR histograms are a means to capture good LDR images, capturing HDR frames is the goal in these methods. One such approach is [1] . It uses the histogram of a previous frame in an HDR video to determine optimal shutter speed sequences to capture the next HDR frame. Its focus lies on finding a trade-off between the amount of well-exposed pixels and the number of exposures. Another technique for determining exposure values in the context of HDR image creation is [2] . Instead of varying the shutter speed, gain is adjusted here. The technique is based on the observation that increasing the gain also increases the SNR of the created HDR image.
EXPOSURE VALUE DETERMINATION
Our system to find suitable exposure values for smartphone photography begins by querying the user's preferences. This is done by letting the user specify a function f (p) over the range of pixel values p = 0, . . . , 255 which describes the desired shape of the image's histogram. The function assigns a value between 0 and 1 to each pixel value p. It expresses the desirability to have pixels with a value of p in the picture that is about to be taken. The system selects a suitable exposure value based on this desirability function f (p). In our implementation, f (p) is defined as a user-adjustable piecewise linear function.
EXPOSURE VALUE DETERMINATION
The user then points the camera at the scene and starts the exposure value algorithm. It begins by temporarily capturing a sequence of images with varying exposure values. Most smartphones are limited with respect to the range and number of exposure values ∆t they permit. An example would be a range of ∆ti = 2 i/k with i = −k, . . . , k for any value of k, i.e., a range between 0.5 and 2 divided into 2k + 1 steps. Due to this limitation, we only use three different values: Two at the extremes and one in the center of the exposure range. We denote them by ∆t − , ∆t 0 and ∆t + . Using them to capture three images results in a dark image I − , a medium image I 0 and a bright one I + respectively. Note that no HDR image is created with our system. Camera motion between the shots thus has little impact on the results and is neglected. The image sequence I − , I 0 , I + is now used to create a high dynamic range histogram spanning the entire brightness range of the scene from the darkest to the brightest area. We refer to the real brightness in the scene as radiance E. In order to create a histogram over radiance, we do not actually need to store an entire HDR image E(x, y).
EXPOSURE VALUE DETERMINATION
The radiance E(x, y) can be calculated for each pixel position (x, y) individually and inserted into the histogram one at a time. We thus drop the parameters x and y in the following and always assume a single pixel position.
EXPOSURE VALUE DETERMINATION
The radiance of a pixel is estimated from an image I by dividing its pixel value by its exposure. We thus obtain one radiance estimateẼ = I/∆t for each of the three images. We denote the three estimates byẼ − ,Ẽ 0 andẼ + . The three estimates should roughly be the same, meaning that the same radiance value in the scene gave rise to three different pixel values under three different exposure settings. The difference betweenẼ − ,Ẽ 0 andẼ + is due to noise and saturation effects when capturing (note that we assume a linear camera response function in our work). It is generally assumed that pixel values p near the center of the [0..255] interval result in a better radiance estimate than values near the extremes. The three estimates are thus weighted by a weighting function w(p) which reflects this characteristic. Examples for weighting functions w(p) can be found in the literature on HDR imaging. The radiance value E for a pixel is then calculated as a weighted average over the three estimates:
EXPOSURE VALUE DETERMINATION
The radiance E is now inserted into the HDR histogram H(j). With the range of shutter speeds known, we also know the range of possible radiance values. The minimum is obtained from a pixel that has a value of 0 even in the longest exposure, that is, Emin = 0/∆t + = 0. The maximum is Emax = 255/∆t − accordingly. This range is subdivided into N histogram bins. Each bin H(j) counts the occurrence of radiance values between Emin + j N (Emax − Emin) and the beginning of bin H(j + 1) for j = 0, . . . , N − 1. The system tests which interval E falls into and increases the corresponding histogram bin by one. Doing this for every pixel position results in a full HDR histogram.
EXPOSURE VALUE DETERMINATION
For the purpose of exposure value determination, we are interested in what the conventional LDR histogram h∆t(p) of an image looks like when it was captured at a specific exposure value ∆t. h∆t(p) can be calculated directly from H(j). This is much faster than actually capturing an image at ∆t. Each bin of h∆t(p) corresponds to a pixel value p between 0 and 255. By dividing each pixel value by the exposure value ∆t, we obtain a radiance estimateẼ = p/∆t. Doing the same test as above,Ẽ can be mapped to an index j in the HDR histogram to which p corresponds. We then simply set h∆t(p) := H(j) for p = 0, . . . , 254. The bin with index 255 contains the sum of all HDR histogram bins with radiance values higher than 254/∆t. This corresponds to saturation of pixels for large brightness values in the scene.
EXPOSURE VALUE DETERMINATION
In this manner, we obtain an LDR histogram for any exposure value ∆t. The sum of its bins does not necessarily equal the total number of pixels anymore due to the way H(j) is sampled. It is thus necessary to normalize the histogram. The system now calculates the coverage C between h∆t(p) and the desired shape of the histogram f (p) (see Figure 1 ):
EXPOSURE VALUE DETERMINATION
We find the exposure value that best matches the user's preference by exhaustive searching. The ∆t to the LDR histogram with the highest coverage with f is selected and finally used to capture the desired image.
CONCLUSIONS
In our technical demo, we present a smartphone app that determines optimal exposure values based on an HDR histogram of the scene. The user can adjust a desired shape for the histogram of the image to be captured. The app then finds the exposure which results in a histogram that best matches the user's preference.
CONCLUSIONS
The most time consuming step in this process is the capturing of the image sequence and the creation of the HDR histogram. It takes about 1.5 seconds. Computation of the LDR histogram and its coverage are very fast operations. Also, the range of values for ∆t is usually small on a smartphone. After HDR histogram creation, calculating the optimal shutter for a desired shape can thus be done without noticeable delay.
CONCLUSIONS
In our experiments, we found that the automatic exposure control in smartphones leads to dissatisfactory images for scenes with difficult lighting conditions (e.g., very bright and very dark areas visible at the same time). In these situations, the subjective image quality according to the user's intention was improved by using our system.

{'paper_id': '14121831', 'title': 'Determining exposure values from HDR histograms for smartphone photography', 's2_url': 'https://api.semanticscholar.org/CorpusID:14121831', 'source': 'S2ORC'}