Convolutional neural network-based regression for
depth prediction in digital holography
Tomoyoshi Shimobaba

Takashi Kakue

Tomoyoshi Ito

arXiv:1802.00664v1 [cs.CV] 2 Feb 2018

Graduate School of Engineering
Graduate School of Engineering
Graduate School of Engineering
Chiba University
Chiba University
Chiba University
1-33 Yayoi-cho, Inage-ku, Chiba, Japan 1-33 Yayoi-cho, Inage-ku, Chiba, Japan 1-33 Yayoi-cho, Inage-ku, Chiba, Japan
shimobaba@faculty.chiba-u.jp
t-kakue@chiba-u.jp
itot@faculty.chiba-u.jp

Abstract—Digital holography enables us to reconstruct objects
in three-dimensional space from holograms captured by an
imaging device. For the reconstruction, we need to know the
depth position of the recoded object in advance. In this study,
we propose depth prediction using convolutional neural network
(CNN)-based regression. In the previous researches, the depth
of an object was estimated through reconstructed images at
different depth positions from a hologram using a certain metric
that indicates the most focused depth position; however, such
a depth search is time-consuming. The CNN of the proposed
method can directly predict the depth position with millimeter
precision from holograms.
Index Terms—digital holography, convolutional neural network, multiple regression, depth prediction

I. I NTRODUCTION
Digital holography is a promising imaging technique because it enables simultaneously measuring the amplitude
and phase of objects, and reconstructing objects in threedimensional (3D) space from a hologram captured by an
imaging sensor [1]. Digital holography can be applied from
microscopic [2] objects (digital holographic microscopy) to
macroscopic objects [3]. For measuring the objects in 3D
space, we need to know the depth position of the recorded
object in advance. Subsequently diffraction is calculated at
that position. When using the angular spectrum method [4], the
diffraction calculation is performed by the following equation:
 


(1)
uz (x, y) = F −1 F uh (x, y) H(µ, ν) ,
where uh (x, y) and uz (x, y) is a hologram and the complex
amplitude in the reconstructed
plane at depth z, and the

operators F · and F −1 · are the Fourier transform and its
inverse transform, respectively. H(µ, ν) is the transfer function
of the angular spectrum method [4].
Without knowing the object’s depth position in advance,
in previous researches, depth search was performed by performing multiple diffraction calculations for different depth
parameters z, and detecting the most focused depth position
by certain focusing metrics.
A number of focusing metrics in digital holography have
been proposed; e.g., entropy-based methods [5]–[7], a Fourier
spectrum-based method [8] and a wavelet-based method [9].
Moreover, the Laplacian ,variance, gradient, and Tamura coefficients of the intensity of a reconstructed image have been

used as focused metrics [8], [10]. Almost all the methods
measure the sharpness of the reconstructed intensity images
and judge the sharpest image as the focused image.
The Tamura coefficient is easier to find global solution of
depth position compared to the Laplacian, variance, gradient
metrics [10]. The coefficient Cz is calculated as
σI
Cz = ¯z ,
(2)
Iz
where Iz = |uz (x, y)|2 is the reconstructed intensity, σI and
I¯ are the standard deviation and average of the intensity,
respectively. In this study, we use the Tamura coefficient for
comparison to the proposed method. The depth search using
this metric is performed as follows: we calculate multiple
reconstructed intensities Iz using the diffraction calculation
equation Eq.(1) and varying changing the depth position z
from z1 to z2 with a step δz. Subsequently, we calculate the
focused metrics Cz for the reconstructed intensities. We can
find the focused depth position at the maximum Cz . However,
such a depth search is time-consuming, because it requires
multiple diffraction calculations.
This study proposes depth prediction using convolutional
neural network (CNN)-based regression. The CNN in the
proposed method can directly predict the depth position with
millimeter precision from holograms, without multiple diffraction calculations.
II. P ROPOSED

METHOD

Recently, deep learning [11] has been intensively investigated and applied in a wide range of research fields. Deep
learning is a type of neural network, but it has deeper layers
than conventional neural networks. Deep learning has been
apllied in the holography, for example, classification [12]–
[14], complex amplitude restoration in digital holographic
microscopy [15], and noise suppression of holographic reconstructed images [16]. Among the many deep neural networks,
CNN demonstrates excellent performance in the field of image
processing, which comprises convolutional layers, pooling
layer, and fully connected layers.
In this study, a continuous depth value is predicted by
inputting hologram information to the CNN of the proposed
method; i.e., the CNN needs to solve a multiple regression

problem. The pioneer work of depth prediction in digital
holography was presented by T. Pitkäaho, A. Manninen and T.
J. Naughton [17], [18]. The difference between the proposed
method and those applied in previous studies [17], [18] is
that in these studies, the depth prediction was solved as a
classification problem, so that the predicted depth becomes a
discrete value. On the other hand, since the proposed method
solves the problem as a multiple regression, the predicted
depth becomes a continuous value.
Figure 1 depicts the CNN structure in the proposed method.

from the dataset “Caltech-256” [19] as original objects of the
holograms and calculated the holograms as inline hologram
from these natural images. The hologram size is 1, 024×1, 024
pixels. The reference light with the wavelength of 633nm is
planar wave. Twenty holograms were captured while moving
a same original object along the depth direction ranging from
0.05m to 0.25m at random δz intervals of [-0.5mm, 0.5mm].

Fig. 1. CNN for predicting focused depth.

The input layer is for inputting hologram information. In
this study, we use two types of hologram information, the raw
interference pattern and the power spectrum of the hologram,
and we will compare the differences in the prediction results
later. The convolution layer performs convolution operations
with the kernel size of 3 × 3 pixels to acquire feature maps of
the input information. The dimension of the first convolution
layer is 128 × 128 × 32 which denotes an input image size
of 128 × 128 pixels and 32 different convolution kernels. All
the convolution layers are connected to activation functions
(ReLU function) and max-pooling layers. The dimensions of
the second, third, and forth convolution layers are 64×64×64,
32 × 32 × 128 and 16 × 16 × 256. The dimension of each fully
connected layer is 2,048. The activation function of the output
layer is a linear function (identity function, i.e., y = x) because
we want to obtain a continuous depth value.
This network is trained by minimizing the loss function,
where we use the mean square error (MSE) between the
(j)
(j)
outputs do of this network and depth values dt included
in a dataset. The loss function (MSE) is defined as
N −1
1 X (j)
(j)
|d − dt |2 ,
e=
N j=0 o

(3)

where the subscript j denotes j-th data in the dataset and N
is the size of the dataset. The CNN is trained using Adam
optimizer [11] with the initial learning rate of 0.0005. The
learning rate is automatically decreased when the MSE is
stagnated.
III. R ESULTS
We prepare two kinds of datasets as illustrated in Fig.2. The
holograms and the power spectra of two objects are presented
as examples. The first dataset consists of raw images of
holograms and their depth values. We use natural images taken

Fig. 2. “Hologram dataset” and “power spectrum dataset.” These datasets
consist of holograms (or spectra) and the corresponding depth values.

Since the holograms were acquired by an image sensor with
1, 024 × 1, 024 pixels, we extracted the 128 × 128 pixels in the
center of the hologram. The reason for reducing the size of
the hologram is to speed up the learning and prediction of the
CNN. Accordingly, it helps to simplify the network structure.
The second dataset consists of the power spectra of the
holograms and the corresponding depth values, as depicted
in Fig.2. We call the dataset “spectrum dataset.” The power
spectra are calculated from the holograms of 1, 024 × 1, 024
pixels, and subsequently, the first quadrant of the calculation
result is extracted and further reduced to 128 × 128 pixels by
linear interpolation.
The hologram and spectrum datasets are prepared for training and validation, respectively. Figure 3 depicts the change of
the loss function for the training and validation datasets with
increasing epoch. As can be seen, in the hologram dataset,
the loss value can only be reduced to 0.00249, which means
that the average depth error is about 50 mm. In contrast,
the loss value of the spectrum dataset for validation reaches
5.245 × 10−5 , which means that the average depth error is
only 7.2 mm.
Figure 4 depicts a hologram with 1, 024 × 1, 024 pixels to
verify the effectiveness of the CNN. The hologram is recorded
from an original object at 0.143 m . We perform the depth
search using the Tamura coefficient (Eq.(2)) ranging from 0.05
m to 0.25 m with the depth step of 1 mm. The coefficient plot
is shown in Fig.5. The plot shows the maximum coefficient at

Fig. 3. Training and validation losses for the hologram and spectrum datasets
with increasing epoch.

Fig. 5. Coefficient plot with increasing the propagation distance of diffraction
calculation.

z = 0.147 m; therefore, in this case, the difference between
the correct depth and the calculated depth is 4 mm. Figure 6
depicts the reconstructed image of the hologram at z = 0.147
m.
Subsequently, the CNN can be used to predict the depth
value of 0.138 m directly from the power spectrum of the
hologram, without the depth search. Figure 7 depicts the
reconstructed image of the hologram at z = 0.138 m. The
difference between the actual depth and the predicted depth is
5 mm.
The calculation time of the CNN is 3.5ms per one power
spectrum on a NVIDIA GeForce 970 GTX GPU; in contrast,
the calculation time of the depth search in Fig.5 is 2,892ms on
the same GPU. Compared with the depth search, the CNN can
greatly speed up the calculation of depth prediction. All the
hologram calculations were performed using our wave optics
library, CWO++ [20].

Fig. 6. Reconstructed image of the hologram at z = 0.147 m estimated by
the maximum value of Tamura coefficient. The difference between the correct
depth and the estimated depth is 4 mm.

IV. C ONCLUSION

Fig. 4. Hologram recorded from an original object at approximately 0.143
m.

We proposed a CNN-based regression for depth prediction
in digital holography. The CNN enables the direct prediction
of the depth value with millimeter precision from the power
spectrum of a hologram. We used a power spectrum that
reduced the size to 1/8th of the original holograms. This
reduction helped simplify the CNN network structure. It will
be also useful for predicting depth value for larger holograms.
In future work, we plan to estimate a depth value with a CNN
regression using optically recorded holograms and predict the
depth values of holograms of multiple recorded objects at
different depth position.

Fig. 7. Reconstructed image of the hologram at z = 0.138m directly
predicted by the CNN-based regression. The difference between the correct
depth and the predicted depth is 5mm.

ACKNOWLEDGMENT
This work was partially supported by JSPS KAKENHI
Grant Number 16K00151.
R EFERENCES
[1] T.-C. Poon, Digital holography and three-dimensional display: Principles and Applications. Springer Science & Business Media, 2006.
[2] M. K. Kim, “Principles and techniques of digital holographic microscopy,” Journal of Photonics for Energy, pp. 018 005–018 005, 2010.
[3] T. Nakatsuji and K. Matsushima, “Free-viewpoint images captured using
phase-shifting synthetic aperture digital holography,” Applied optics,
vol. 47, no. 19, pp. D136–D143, 2008.
[4] J. W. Goodman, Introduction to Fourier optics. Roberts and Company
Publishers, 2005.
[5] Y. Han and Q. Yue, “Laplacian differential reconstruction of one in-line
digital hologram,” Optics Communications, vol. 283, no. 6, pp. 929–931,
2010.
[6] Z. Ren, N. Chen, A. Chan, and E. Y. Lam, “Autofocusing of optical
scanning holography based on entropy minimization,” in Digital Holography and Three-Dimensional Imaging. Optical Society of America,
2015, pp. DT4A–4.
[7] S. JIAO, P. W. M. Tsang, T.-C. Poon, J.-P. Liu, W. Zou, and X. Li, “Enhanced autofocusing in optical scanning holography based on hologram
decomposition,” IEEE Transactions on Industrial Informatics, 2017.
[8] P. Langehanenberg, B. Kemper, D. Dirksen, and G. Von Bally, “Autofocusing in digital holographic phase contrast microscopy on pure phase
objects for live cell imaging,” Applied optics, vol. 47, no. 19, pp. D176–
D182, 2008.
[9] M. Liebling and M. Unser, “Autofocus for digital fresnel holograms
by use of a fresnelet-sparsity criterion,” JOSA A, vol. 21, no. 12, pp.
2424–2430, 2004.
[10] P. Memmolo, C. Distante, M. Paturzo, A. Finizio, P. Ferraro, and
B. Javidi, “Automatic focusing in digital holography and its application
to stretched holograms,” Optics letters, vol. 36, no. 10, pp. 1945–1947,
2011.
[11] I. Goodfellow, Y. Bengio, and A. Courville, Deep learning. MIT press,
2016.

[12] T. Shimobaba, N. Kuwata, M. Homma, T. Takahashi, Y. Nagahama,
M. Sano, S. Hasegawa, R. Hirayama, T. Kakue, A. Shiraki et al., “Convolutional neural network-based data page classification for holographic
memory,” Applied optics, vol. 56, no. 26, pp. 7327–7330, 2017.
[13] N. Muramatsu, C. W. Ooi, Y. Itoh, and Y. Ochiai, “Deepholo: recognizing 3d objects using a binary-weighted computer-generated hologram,”
in SIGGRAPH Asia 2017 Posters. ACM, 2017, p. 29.
[14] S.-J. Kim, B. Zhao, H. Im, J. Min, N. Choi, C. M. Castro, R. Weissleder,
H. Lee, and K. Lee, “Deep-learning based hologram classification for
molecular diagnostics,” bioRxiv, p. 192559, 2017.
[15] Y. Rivenson, Y. Zhang, H. Gunaydin, D. Teng, and A. Ozcan, “Phase
recovery and holographic image reconstruction using deep learning in
neural networks,” arXiv preprint arXiv:1705.04286, 2017.
[16] T. Shimobaba, Y. Endo, R. Hirayama, Y. Nagahama, T. Takahashi,
T. Nishitsuji, T. Kakue, A. Shiraki, N. Takada, N. Masuda et al.,
“Autoencoder-based holographic image restoration,” Applied Optics,
vol. 56, no. 13, pp. F27–F30, 2017.
[17] T. Pitkäaho, A. Manninen, and T. J. Naughton, “Performance of
autofocus capability of deep convolutional neural networks in digital
holographic microscopy,” in Digital Holography and Three-Dimensional
Imaging. Optical Society of America, 2017, pp. W2A–5.
[18] T. Pitkäaho, A. Manninen, and T. J. Naughton, “Focus classification
in digital holographic microscopy using deep convolutional neural
networks,” in European Conference on Biomedical Optics. Optical
Society of America, 2017, p. 104140K.
[19] G. Griffin, A. Holub, and P. Perona, “Caltech-256 object category
dataset,” 2007.
[20] T. Shimobaba, J. Weng, T. Sakurai, N. Okada, T. Nishitsuji, N. Takada,
A. Shiraki, N. Masuda, and T. Ito, “Computational wave optics library
for c++: CWO++ library,” Comput. Phys. Commun., vol. 183, no. 5, pp.
1124–1138, 2012.

