Title: Deep Speaker Verification: Do We Need End to End?,
Abstract: End-to-end learning treats the entire system as a whole adaptable black box,
which, if sufficient data are available, may learn a system that works very
well for the target task. This principle has recently been applied to several
prototype research on speaker verification (SV), where the feature learning and
classifier are learned together with an objective function that is consistent
with the evaluation metric. An opposite approach to end-to-end is feature
learning, which firstly trains a feature learning model, and then constructs a
back-end classifier separately to perform SV. Recently, both approaches
achieved significant performance gains on SV, mainly attributed to the smart
utilization of deep neural networks. However, the two approaches have not been
carefully compared, and their respective advantages have not been well
discussed. In this paper, we compare the end-to-end and feature learning
approaches on a text-independent SV task. Our experiments on a dataset sampled
from the Fisher database and involving 5,000 speakers demonstrated that the
feature learning approach outperformed the end-to-end approach. This is a
strong support for the feature learning approach, at least with data and
computation resources similar to ours.