title: Exploring British Accents: Modelling the Trap-Bath Split with Functional Data Analysis
authors: Aranya Koshy Shahin Tavakoli
text:  
  Sum Rate Maximization for Intelligent Reflecting Surface Assisted Terahertz Communications
      Yijin Pan, Kezhi Wang, Cunhua Pan, Huiling Zhu and Jiangzhou Wang, Fellow, IEEE 
  
  	Y. Pan are with the National Mobile Communications Research Laboratory, Southeast University, Nanjing 211111, China. She is also with School of Engineering and Digital Arts, University of Kent, UK. Email: panyj@seu.edu.cn, y.pan@kent.ac.uk.
  	K. Wang is with the Department of Computer and Information Sciences, Northumbria University, UK. Email: kezhi.wang@northumbria.ac.uk.
  	C. Pan is with the School of Electronic Engineering and Computer Science, Queen Mary, University of London, UK. Email: c.pan@qmul.ac.uk.
  	J. Wang and H. Zhu are with the School of Engineering and Digital Arts, University of Kent, UK. Email: J.Z.Wang@kent.ac.uk, H.Zhu@kent.ac.uk.
      December 30, 2023
  =======================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================
  
  empty
  
  
  1.45
  
  
  
  
  Abstract
   
  
  
  
  The sound of our speech is influenced by the places we come from. Great Britain contains a wide variety of distinctive accents which are of interest to linguistics. In particular, the “a” vowel in words like “class” is pronounced differently in the North and the South. Speech recordings of this vowel can be represented as formant curves or as mel-frequency cepstral coefficient curves. Functional data analysis and generalised additive models offer techniques to model the variation in these curves. Our first aim is to model the difference between typical Northern and Southern vowels /æ/ and /A/, by training two classifiers on the North-South Class Vowels dataset[] collected for this paper
  . Our second aim is to visualise geographical variation of accents in Great Britain. For this we use speech recordings from a second dataset, the British National Corpus (BNC) audio edition
  . The trained models are used to predict the accent of speakers in the BNC, and then we model the geographical patterns in these predictions using a soap film smoother. This work demonstrates a flexible and interpretable approach to modelling phonetic accent variation in speech recordings.
  
  
  
    
  Keywords: Formants, functional principal component analysis, logistic regression,  phonetics,  MFCC.
  
  
  1.5
  
  
  
  
  § PHONETICS
   
  
  Phonetic variation in speech is a complex and fascinating phenomenon. The sound of our speech is influenced by the communities and groups we belong to, places we come from, the immediate social context of speech, and many physiological factors. There is acoustic variation in speech due to sex and gender specific differences in articulation <cit.>, age <cit.>, social class and ethnicity <cit.>, and individual idiosyncrasies of sound production <cit.>. This linguistic variation is relevant to many fields of study like anthropology, economics and demography
  <cit.>, and has connections to the study of speech production and perception in the human brain. It helps us understand how languages developed in the past, and the evolutionary links that still exist between languages today <cit.>. Modelling phonetic variation is also important for many practical applications, like speech recognition and speech synthesis. In this work, we study one source of variation in particular: geographical accent variation.
  
  
  To describe phonetic variation conveniently, in his seminal work Accents of English, <cit.> introduced lexical sets, which are groups of words containing vowels that are pronounced the same way within an accent. The trap lexical set contains words like trap, cat and man, and the bath lexical set contains words like bath, class and grass. In Northern English accents both trap and bath words use the `short a' vowel /æ/. In Southern English accents trap words use /æ/ and bath words use the  `long a' vowel /A/; this is known as the trap-bath split. 
  
  The trap-bath split is one of the most well studied geographical accent differences. The geographical accent variation in sounds like these has historically been studied using written transcriptions of speech from surveys and interviews by trained linguists. These were used to construct isogloss maps (see Figure <ref>) to visualise regions having the same dialect. <cit.> explain that in reality these isoglosses are not sharp boundaries, and they are drawn to show only the most prominent linguistic variation in a region for the sake of simplicity. The boundaries are also constantly moving and changing over time.
  
  
  
  
  More recently, advances in statistical methods and technology have allowed accent variation to be modelled by directly using audio recordings of speech. A sound can be represented as a set of smooth curves, and functional data analysis <cit.> offers techniques to model variation in these curves. This work demonstrates one such approach, in which we analyse variation in vowel sounds using techniques from FDA and generalised linear models.
  
  
  This paper has two main contributions. The first contribution is to use functional data analysis to classify vowels by directly using speech recordings: we demonstrate two approaches for classifying bath vowels as Northern or Southern. The first approach models variation in formant curves (see Section <ref>) using a functional linear model. The second approach models variation in mel-frequency cepstral coefficient (MFCC) curves (see Section <ref>) through penalised logistic regression on functional principal components, and it can be used to resynthesise vowel sounds in different accents, allowing us to “listen to the model”. Both approaches classify accents using the temporal dynamics of the MFCC or formant curves in sounds. These two classifiers were trained using a dataset of labelled audio recordings[] that was collected for this paper in an experimental setup <cit.>.
  The second contribution is to construct maps that visualise geographic variation in the bath vowel that can be attributed to typical Northern and Southern accent differences, using a soap film smoother. For this we use the audio BNC dataset <cit.>, which is a representative sample of accents in Great Britain. The resulting maps show a geographical variation in the vowel similar to what is seen in isogloss maps like Figure <ref>.
  
  The paper is structured as follows. In Section <ref>, we introduce two ways of representing vowel sounds as multivariate curves. Section <ref> introduces the two datasets used in this analysis, and the preprocessing steps involved. Section <ref> gives the two models for classifying bath vowels, and Section <ref> presents the maps constructed to visualise geographical accent variation. We conclude with a discussion of the results in Section <ref>.
  
  
  
  § SOUND AS DATA OBJECTS
   
  
  Sound is a longitudinal air pressure wave. Microphones measure the air pressure at fixed rates, for example at 16 kHz (Hz is a unit of frequency representing samples per second). The waveform of the vowel in the word “class” in Figure <ref> shows this rapidly oscillating air pressure wave as measured by a microphone. This signal can be transformed in several ways to study it; for example as a spectrogram, formants, or mel-frequency cepstral coefficients (MFCCs), see Sections <ref>, <ref> and <ref>. 
  
  
  
  
  
  
  
   §.§ Spectrograms
  
  
  
  We begin by defining the spectrogram of a sound. A spectrogram is a time-frequency representation of a sound: it reveals how the most prominent frequencies in a sound change over time. To define it precisely, let us denote the sound wave as a time series {s(t): t = 1, …, T}, where s(t) is the deviation from normal air pressure at time t. We can define s(t)=0 for t≤ 0 or t>T. Let w: ℝ→ℝ be a symmetric window function which is non-zero only in the interval [-M/2,M/2] for some M<T. The Short-Time Fourier Transform of {s(t)}_t=1^T is computed as
  
      STFT(s)(t, ω)    = ∑_u=-∞^∞ s(u)w(u-t)exp(-iω u) 
          =  ∑_u=1^T s(u)w(u-t)exp(-iω u),
  
  for t=1,…,T, and ω∈{2π k/N: k=0, …, N-1} for some N≥ T which is a power of 2. The window width M is often chosen to correspond to a 20 ms interval. 
  The spectrogram of {s(t)}_t=1^T is then defined as
  
      Spec(s)(t, ω)     = |STFT(s)(t, ω)|^2.
  
  At a time point t, the spectrogram shows the magnitude of different frequency components ω in the sound. Figure <ref> shows spectrograms of recordings of different vowels, with time on the x-axis, frequency on the y-axis, and colour representing the amplitude of each frequency. The dark bands are frequency peaks in the sound, which leads us to the concept of formants.
  
  
  
  
   §.§ Formants
  
  
  Formants are the strongest frequencies in a vowel sound, observed as high-intensity bands in the spectrogram of the sound. By convention they are numbered in order of increasing frequency, F_1, F_2, …. 
  
  
  
  Formants are produced by the resonating cavities and tissues of the vocal tract <cit.>. The resonant frequencies depend on the shape of the vocal tract, which is influenced by factors like rounding of the lips, and height and shape of the tongue (illustrated in Figure <ref>). The pattern of these frequencies is what distinguishes different vowels. They are particularly important for speech perception because of their connection to the vocal tract itself, and not the vocal cords. Listeners use formants to identify vowels even when they are spoken at different pitches, or when the vowels are whispered and the vocal cords don't vibrate at all <cit.>. One can also sometimes “hear” a person smile as they speak, because the act of smiling changes the shapes of the vocal cavities and hence the formants produced <cit.>.
  
  
  
  
   
  
  
  
   §.§ Mel-Frequency Cepstral Coefficients
  
  
  
  Mel-frequency cepstral coefficients (MFCCs) are a further transformation of the spectrogram, and are often used in speech recognition and speech synthesis. The way they are constructed is related to how the human auditory system processes acoustic input; in particular, how different frequency ranges are filtered through the cochlea in the inner ear. This filtering is the reason humans can distinguish between low frequencies better than high frequencies. MFCCs roughly correspond to the energy contained in different frequency bands, but are not otherwise easily interpretable.
  
  
  There are many variants of MFCCs; we use the one from <cit.> which allow for high fidelity sound resynthesis.
  
  MFCCs are computed in two steps as follows <cit.>. First the mel-spectrogram is computed from the spectrogram, using a mel scale filter bank with F filters (b_f,k)_k=0,…,N-1, f=0, …, F. The mel scale is a perceptual scale of pitches, under which pairs of sounds that are perceptually equidistant in pitch are also equidistant in mel units. 
  This is unlike the linear Hz scale, in which a pair of low frequencies will sound further apart than an equidistant pair of high frequencies. 
  The mapping from Hz (f) to mels (m) is given by m=2595 log_10(1+f/700), shown in Figure <ref>. The mel-spectrogram is defined as
  
  
      MelSpec(s)(t, f)    = ∑_k=0^N-1Spec(s)(t, 2π k/N)b_f,k.
  
  
  
  
  In the second step, we take the inverse Fourier transform of the logarithm of this mel-spectrogram. The first M resulting coefficients are the MFCCs,
  
      MFCC(s)(t, m)    = 1/F∑_f=0^Flog(MelSpec(s)(t, f)) exp( i2π (m-1)f/F+1).
  
  At each time point t we have M MFCCs. We use the  software <cit.> to extract MFCCs, which uses M=40 at each time point. Thus we represent each vowel sound by 40 MFCC curves. 
  
  
  
  
  Formants are a low-dimensional summary of the original sound which allow interpretation of the vocal tract position. MFCCs retain a lot of information about speech sounds and do not simplify the representation in an immediately interpretable way, but the model with MFCCs allows us to resynthesise bath vowels along the /æ/ to /A/ spectrum. MFCCs and formants therefore have different strengths and limitations for analysis, depending on the goal. In this paper we demonstrate classifiers using both representations. 
  
  
  Regardless of whether we work with vowel formants or MFCCs, we can view the chosen sound representation as a smooth multivariate curve over time, X(t) ∈ℝ^d, where t ∈ [0,1] is normalised time. In practice we assume X(t) is observed with additive noise due to differences in recording devices and background noise in the recording environment. 
  
  
  
  § DATA SOURCES
   
  
  In this section we describe the two data sources used in this paper.
  
  
  
  
  
   §.§ North-South Class Vowels
   
  
  The North-South Class Vowels <cit.> 
  
  
  
  dataset is a collection of 400 speech recordings of the vowels /æ/ and /A/ that distinguish stereotypical Northern and Southern accents in the bath lexical set. The vowels were spoken by a group of 4 native English speakers (100 recordings per speaker).
  It was collected in order to have a high-quality labelled dataset of the /æ/ and /A/ vowel sounds in bath words.
  The NSCV dataset was collected with ethical approval from the Biomedical and Scientific Research Ethics Committee of the University of Warwick.
  
  The speech recordings were collected in an experimental setup. The speakers were two male and two female adults between the ages of 18 and 55.  In order to participate they were required to be native English speakers but were not required to be proficient in Southern and Northern accents. 
  
  To access the vowels, they were shown audio recordings as pronunciation guides, and example rhyming words such as `cat' for the /æ/ vowel and `father' for the /A/ vowel. They were allowed to practice using the two vowels in the list of words, before being recorded saying a list of words using both vowels. The words were class, grass, last, fast, and pass. Each word was repeated 5 times using each vowel, by each speaker. The speech was simultaneously recorded with two different microphones.
  
  
  The purpose of this dataset is to demonstrate a method of training accent classification models. By using vowels as a proxy for accent, it allows us to train models to distinguish between Northern and Southern accents, to the extent that they differ by this vowel. Using two microphones and having the same speaker producing both vowels allows us to train models that are robust to microphone and speaker effects. Despite the small number of speakers in this dataset, we are still able to classify vowels with high accuracy and resynthesise vowels well. A limitation of the dataset is that the speakers were not required to be native speakers of both Northern and Southern accents or have any phonetic training.
  
  
  
   §.§ British National Corpus
   
  
  The audio edition of the British National Corpus (BNC) is a collection of recordings taken across the UK in the mid 1990s, now publicly available for research <cit.>. A wide range of people had their speech recorded as they went about their daily activities, and the audio recordings were annotated (transcriptions of the conversations, with information about the speakers). From this corpus we analyse utterances of the following words from the bath lexical set, which we call the “class” words: class, glass, grass, past, last, brass, blast, ask, cast, fast, and pass.
  
  Among the sound segments in the BNC labelled as a “class” word, not all of them do correspond to a true utterance of a “class” word by a British speaker[], and some are not of good quality. Some sounds were removed from the dataset using the procedure described in Appendix <ref>. 
  
  The resulting dataset contains 3852 recordings from 529 speakers in 124 locations across England, Scotland and Wales. Figure <ref> shows the number of sounds and speakers at each location. Some speakers were recorded at multiple locations, but 94% of them have all their recording locations within a 10 kilometre radius. 88% of all speakers only have one recording location in this dataset.
  
  
  
  This dataset captures a wide range of geographical locations and socio-economic characteristics, and speakers were recorded in their natural environment. It has, however, some limitations for our analysis. For example, we do not know the true origin of a speaker, so unless the metadata shows otherwise, we must assume that speakers' accents are representative of the location where they were recorded. There are very few speech recordings available from the North, especially Scotland. The timestamps used to identify word boundaries are often inaccurate, and the sound quality varies widely between recordings, due to background noise and the different recording devices used.
  
  
  
   §.§ Transforming sounds into data objects
   
  
  
  
  Each speech recording in the BNC and NSCV datasets was stored as a mono-channel 16 kHz  file. The raw formants were computed using the  R package <cit.>. At each single time point the first four formants were computed, and this is done at 200 points per second. A sound of length 1 second is thus represented as a 200×4 matrix, where each column corresponds to one formant curve. For each vowel sound, raw MFCCs were extracted using the  software <cit.>, which also computes them at 200 points per second. Hence a sound of length 1 second would be represented as a 200×40 matrix, where each column represents one MFCC curve. 
  
  
  
  We smooth the raw formants and raw MFCCs in order to remove unwanted variation due to noise, and to renormalise the length of the curves by evaluating each smoothed curve at a fixed number of time points <cit.>. 
  
  Assuming a signal plus noise model on the raw formants and raw MFCCs, we smooth and resample them on an equidistant grid of length T=40. Since the raw formants exhibit large jumps that are physiologically implausible, we smooth them using robust loess <cit.> with smoothing parameter l=0.4 and using locally linear regression. The raw MFCCs are less rough, and we smooth them using cubic splines <cit.>  with knots chosen at each point on the time grid and smoothing parameter chosen by cross-validation for each curve. We have used T=40 in this analysis because it captures the main features while not inflating the dataset too much. We do not model vowel duration, which also depends on other factors, such as speech context <cit.>. Other implementations and smoothing methods could be used here, such as the R package  for smoothing MFCCs with cubic splines, and robust smoothing for formants using the scaled t family.
  
  
  
  
  
  
  
  Finally, we perform an alignment step to reduce misalignments between NSCV curves and BNC curves. This is necessary because the BNC speech recordings often have inaccurate timestamps and this can cause their vowels to be misaligned with the NSCV curves. Since we classify BNC vowels using models trained on NSCV curves, these misalignments can cause inaccuracies in the predictions. We consider the differences in relative timing of the vowel in the sound to be due to a random phase variation; alignment or registration of curves allows us to reduce the effect of this phase variation <cit.>. We use the approach of <cit.>, where the Fisher–Rao metric distance between two curves is minimised by applying a nonlinear warping function to one of the curves. 
  
  The first MFCC curve (MFCC 1) of each sound contains the volume dynamics. To align NSCV vowels, we first align all NSCV MFCC 1 curves together. These warping functions are then applied to the formant curves and other MFCC curves from the same vowels, since they come from the same underlying sounds. For each BNC vowel, we first align its MFCC 1 curve to the mean aligned NSCV MFCC 1 curve, and then use the obtained warping function to align all the other MFCC curves and formant curves from the same vowel. Alignment was performed using the R package  <cit.>, and the preprocessing steps are summarised in Figure <ref>.
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  § CLASSIFYING ACCENTS
   
  
  In this section, we will present two models for classifying bath vowels as Southern or Northern.  
  
  
  
  
   §.§ Modeling formants
   
  
  
  Our first task is to build a classifier to classify bath vowels as Northern or Southern. The model uses the fact that the first two formants F_1 and F_2 are known to predominantly differentiate vowels, and higher formants do not play as significant a role in discriminating them <cit.>. It has been suggested that the entire trajectory of formants are informative even for stationary vowels like the bath vowels, and they should not be considered as static points in the formant space <cit.>; (see also the discussion in Section <ref>). This suggests the use of formant curves as functional covariates when modelling the vowel sounds. 
  Since the dynamic changes in the formant curves are not drastic, we do not believe there are time-localised effects of the formants, so we use the entire formant curve as a covariate with a roughness penalty. Due to the nested structure of the NSCV corpus with 100 speech recordings from each speaker, we also include a random effect term to account for variation between speakers.
  
  
  
  Now we can propose the following functional logistic regression model to classify accents:
  
  
  
  
      (p_ij) = β_0 + ∫_0^1F_2ij(t)β_1(t)dt + γ_j,
  
  
  
  
  
  
  
  where p_ij is the probability of sound i from speaker j being Southern, F_2ij(t) is the value of the F_2 curve at time t for sound i from speaker j, and γ_j ∼ N(0, σ_s^2) is a random effect for speaker j. The functional covariate contributes to the predictor through a linear functional term. The integral is from 0 to 1 since we have normalised the length of all sounds during preprocessing. The function β_1(t) is represented with a cubic spline with knots at each time point on the grid, and its “wiggliness” is controlled by penalising its second derivative. Model selection was done by comparing the adjusted AIC <cit.> to decide which other terms should be included in the model. Further details from the model selection procedure are given in Appendix <ref>, where we also consider simpler non-functional models. The model was fitted using the  package in R <cit.>.
  
  The fitted coefficient curve β̂_1(t), shown in Figure <ref>, reveals that middle section of the F_2 curve is important in distinguishing the vowels. A lower F_2 curve in this region indicates a Northern /æ/ vowel. From a speech production perspective, this corresponds to the Northern vowel being more “front”, which indicates that the highest point of the tongue is closer to the front of the mouth, compared to the Southern vowel. 
  The point estimate for β_0 is 328.0 (p-value = 0.267, 95% CI [-250.85, 906.87]). The variance component explained by the speaker random effects is σ̂_s^2 = 0.006 (p-value = 0.776). 
  
  
  
  
  
  This model assigns a “probability of being Southern” to a given vowel sound, by first aligning the sound to the mean NSCV sound using MFCC 1, and then plugging its formants into (<ref>). We classify a vowel sound as Southern if its predicted probability of being Southern is higher than 0.5. 
  
  We can estimate the classification accuracy of this model through cross-validation. The model was cross-validated by training it on 3 speakers and testing on the fourth speaker's vowels, and repeating this 4 times by holding out each speaker in the dataset. Using a random split of the data instead would lead to overestimated accuracy, because different utterances by the same speaker cannot be considered independent. The cross-validated accuracy is 96.75%, and the corresponding confusion matrix is shown in Table <ref>. We can also compare the performance of this model for different classification thresholds, using the ROC curve in Figure <ref>.
  
  
   
  
  
  
  
  
   §.§ Modeling MFCCs
   
  
  We will now present another approach to classifying vowel sounds, which uses the MFCC curves obtained from each vowel recording. We have 40 smoothed MFCC curves for each sound. 
  
  Unlike with formants, we do not have prior knowledge about which curves contain information about the vowel quality. Additionally, since MFCC curves contain information about different parts of the frequency spectrum, they are not independent and the co-variation between curves is important. For example, setting an MFCC curve (or a region of the curve) to a constant value distorts the resulting sound. Hence a multivariate functional regression approach with ℓ_1 penalty to remove certain curves from the model would not be appropriate, and we need to incorporate information from potentially all the MFCC curves in our model. The problem of concurvity between MFCC curves can also destabilise the resulting coefficient curve estimates in such an approach. 
  
  Interpreting the shapes of the curves is also not as useful here since MFCC trajectories do not have a physical interpretation as formants do. We are more interested in the model's ability to resynthesise vowels by capturing as much relevant information about vowel quality as possible. Hence we use functional principal components analysis to capture the co-variation of the MFCC curves. This step essentially generates new features by reparametrising the MFCC curves, which we can then use to fit the classification model.
  We select the most informative functional principal components to be in the model through ℓ_1 penalisation.
  
  
  
  
  
  
    §.§.§ Functional Principal Component Analysis
  
  
  Functional principal component analysis <cit.> is an unsupervised learning technique which identifies  the different modes of variation in a set of observed smooth curves {X_i: [0,1] →ℝ,  i = 1 ,…, n}. It is very similar to standard principal component analysis, except that the variables are curves instead of scalar features, and each functional principal component (FPC) is also a curve instead of a vector.  
  
  Assuming that the curves {X_i} are centred, the kth FPC is a smooth curve φ_k:[0,1] →ℝ which maximises
  
      1/n∑_i=1^n( ∫φ_k(t) X_i(t) dt ) ^2,
  
  subject to ∫φ_k(t)^2 dt = 1 and ∫φ_k(t)φ_j(t)dt = 0 for all j < k; there is no constraint for k=1. The functional principal component score (FPC score) of curve i with respect to principal component φ_k is s_ik = ∫φ_k(t)X_i(t)dt. 
  
  In multivariate FPCA, each observation is a curve in ℝ^M, and the set of observations is { X_i=(X_i^(1), X_i^(2), …, X_i^(M)): [0,1] →ℝ^M,  i = 1 ,…, n}. Amongst the existing variants of multivariate FPCA <cit.>, we use the following one:
  assuming that the curves { X_i} are centred, the kth FPC is a smooth multivariate curve, defined as φ_k = (φ_k^(1), φ_k^(2), …, φ_k^(M)):[0,1] →ℝ^M which maximises
  
      1/n∑_i=1^n∑_j=1^M( ∫φ_k^(j)(t) X_i^(j)(t) dt)^2
  
  subject to ∑_j=1^M ∫ [ φ_k^(j)(t) ]^2 dt = 1 and ∑_j=1^M∫φ_k^(j)(t)φ_l^(j)(t)dt = 0 for all l < k. The k-th FPC score of X_i is defined as s_ik = ∑_j=1^M ∫φ_k^(j)(t) X_i^(j)(t) dt.
  
  In our case, the curves { X_i} are the MFCC curves with M=40. Each curve X_i discretised on a grid of T equally spaced time points, yielding a T × M matrix, which is then transformed by stacking the rows into a vector in ℝ^MT. The whole dataset is then represented as an n × MT matrix, which contains observations as rows. The (discretised) FPCs and their scores can therefore be directly computed using a standard implementation of (non-functional) PCA, such as  in R <cit.>. 
  
  Before performing the FPCA we centre each MFCC 1 curve at zero, because the average level of MFCC 1 mainly contains differences in the overall volume of the sound, which is influenced by factors other than accent. Centring the curve at zero retains the volume dynamics in the vowel while normalising the overall volume between sounds. Since there are 400 observations in the NSCV training data, we can decompose the MFCC curves into (at most) 400 functional principal components. The first 25 eigenvalues of the FPCs obtained are plotted in Figure <ref>.
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
    §.§.§ ℓ_1-Penalised Logistic Regression
  
  
  ℓ_1-penalised logistic regression <cit.> can be used for binary classification problems when we have many covariates (here we have p=400 FPC scores which we could include in the model, which corresponds to a reparametrisation of the MFCC curves without losing any information). Through the penalisation and model fitting procedure, a smaller subset of covariates are chosen in the final model.
  
  The model is the same as for the usual logistic regression: if Y is a Bernoulli random variable and X∈ℝ^p is its covariate vector, the model is
  
      ( ℙ(Y = 1 | X = x) ) = β_0 + β^𝖳x,
  
  but it is fitted with an added ℓ_1 penalty on the regression coefficients to deal with high-dimensionality, which encourages sparsity and yields a parsimonious model.
  
  In our setting, 
  if y_i = 1 if sound i is Southern, y_i=0 if it is Northern, and x_i ∈ℝ^400 is a vector of its 400 FPC scores, PLR is fitted by solving
  
  
      (β̂_̂0̂, β̂) = max_β_0, β∑_i=1^n (y_i (β_0 + β^𝖳 x_i) - log(1 + e^β_0+ β^𝖳 x_i)) - λ∑_j=1^p|β_j |,
  
  where λ≥ 0 is a penalty weight. 
  
  Notice that the first term in (<ref>) is the usual log-likelihood, and the second term is an ℓ_1 penalty term. The penalty λ is chosen by 10-fold cross-validation.
  
  
  
  
  
  A new sound with FPC scores vector x_* is assigned a “probability of being Southern” of ( β̂_0 + β̂^𝖳x_* ), where 
  
  (·) is the inverse logit function. We classify the sound as Southern if ( β̂_0 + β̂^𝖳x_* ) ≥ 0.5.
  
   
  We can estimate the accuracy of the model by cross-validating using individual speakers as folds, as in the functional linear model of Section <ref>. Within each training set, we first perform the FPCA to obtain the FPCs and their scores. Then we cross-validate the penalised logistic regression model to find the optimal penalty λ, and retrain on the whole training set with this λ. Finally, we project the test speaker's sounds onto the FPCs from the training set to obtain the test FPC scores, and use them to classify the vowel of each sound using the predicted probabilities from the trained model. This process is repeated holding out each speaker in turn. The cross-validated accuracy of this model is 95.25%. The confusion matrix is shown in Table <ref>, and the ROC curve is shown in Figure <ref>. 
  
  To fit the full model, we use the entire dataset to cross-validate to choose the best λ, and then refit on the entire dataset using this penalty. The entries of β are essentially weights for the corresponding FPCs. By identifying the FPC scores which have nonzero coefficients, we can visualise the weighted linear combination of the corresponding FPCs which distinguish Northern and Southern vowels. In total 10 FPCs had nonzero weights, and all of the chosen FPCs were within the first 20. A plot of the first 25 coefficient values is given in Figure <ref>.
  
  
  
  
  
  
  
  
  
  
  
  
    §.§.§ Resynthesising vowel sounds
  
  
  
  
  The combined effect of the functional principal components that are predictive of accent is given by the function 
  
      ∑_k=1^400β̂_kφ̂_̂k̂: [0,1] →ℝ^40.
  
  
  Discretising this function on an equispaced grid of T points yields a T × 40 matrix, which can be visualised (Figure <ref>), or interpreted as a set of MFCC curves (Figure <ref>).
  
  
  This MFCC matrix captures the difference between the /æ/ and /A/ vowels. Since MFCCs can be used to synthesise speech sounds, we can now make a given bath vowel sound more Southern or Northern, through the following procedure:
  
  We first extract the MFCCs for the entire utterance of a bath word, as a T × 40 matrix where T is determined by the length of the sound. With manually identified timestamps we find the T_v rows of this matrix which correspond to the vowel in the word. We align MFCC 1 of this vowel to the mean NSCV MFCC 1 curve, to obtain the optimal warping function for the sound. The MFCC matrix in Figure <ref> is `unwarped' using the inverse of this warping function, resampled at T_v equidistant time points, and padded with T - T_v rows of zeroes corresponding to the rest of the sound's MFCCs (which we do not change). We can then add multiples of this T×40 matrix to the original sound's MFCC matrix and synthesise the resulting sounds using  <cit.>. Adding positive multiples of the matrix makes the vowel sound more Southern, while subtracting multiples makes it sound more Northern. In the supplementary material we provide audio files with examples of this:  contains the word “blast” uttered in a Southern accent and perturbed towards a Northern accent, and  contains the word “class” uttered in a Northern accent and perturbed towards a Southern accent. Both of these original vowels were new recordings, and not from the NSCV corpus.
  
  
  
  
  
  
  
  
  
  
  
  
  § MODELLING GEOGRAPHIC VARIATION
   
  
  In this section we demonstrate an approach for visualising the trap–bath split by combining data from the BNC with the trained accent classifiers described in Sections <ref> and <ref>. For each BNC speaker we predict the probability of their vowel sound being Southern (using in turn the formant model and the MFCC model), and then smooth the predicted probabilities spatially using a soap film smoother.
  
  The BNC bath vowels contain more variation than the NSCV dataset. This is partly because of more natural variation in conversational speech, as well as other factors like poor quality of some recordings and background noise. The BNC recordings also contain whole words and not only the vowel portion of the utterance. The timestamps for word boundaries are often inaccurate and many sounds are either a partial word, or contain parts of other words or speech from other speakers. It is hard to automatically detect the vowel portions within these recordings. We address this issue through the alignment step described in Section <ref> to align each sound to the NSCV using the mean NSCV MFCC 1 curve.
  
  A single representative sound can be constructed for each speaker by taking an average of these aligned formant and MFCC curves from the speaker's utterances. By resynthesising the sound of the average MFCC curves, we can hear that it retains the quality of a bath vowel, so we use these average MFCCs and formants as representative of each speaker's vowel sound. For each speaker we obtain two predicted probabilities of their accent being Southern (one based on the formants, and one on the MFCCs), using models of Sections <ref> and <ref>.  Notice that for each speaker, plugging this average sound's formants (MFCCs) into the trained models of Sections <ref> (Section <ref>) yields the same predicted logit probability as if we averaged the logit probabilities from each sound's aligned formants (aligned MFCCs). The averaging step used to get speaker-specific probabilities ensures that the model is not unduly influenced by individual speakers who have many recordings at one location, while also reducing the predicted probability uncertainties. Where a speaker has recordings at multiple locations, we attribute their average sound to the location with most recordings.
  
  
  
  
  At each location (, ) in Great Britain, we denote by f(, ) the logit of the expected probability of a randomly chosen person's accent being Southern. 
  We will estimate this surface using a spatial Beta regression model:
  
  
  
  
  
  
  
  
  
  
  
  
      p_ij   iid∼   Beta(μ_i ν, ν (1-μ_i)),    j ∈{1, …, n_i}
      (μ_i)    =    f(_i, _i),
  
  where  p_ij∈ [0,1] is the predicted probability of the j-th speaker's accent at location  (_i, _i) being Southern, j=1,…, n_i.
  The surface f is estimated using a soap film smoother within the geographic boundary of Great Britain. 
  A single value of ν > 0 is estimated for all observations, as in GLMs. Notice that (f(_i,_i)) = μ_i = 𝔼(p_ij)  ∈ [0,1] represents the expected probability of the accent of a randomly chosen person being Southern at location (_i, _i).  
  
  One may instead consider fitting a linear model directly on the estimated linear predictor scores obtained from the formant or MFCC models. This linear approach would not be robust to the estimated linear predictor taking large values (which is the case with our data), even though the associated probabilities are essentially equal to one (or zero). Our approach alleviates this by smoothing predictions on the probability scale which makes it less influenced by outliers. Link functions other than the logit could also be used in alternative approaches.
  
  Let us now recall the soap film smoother.
  The soap film smoother <cit.> is a nonparametric solution to spatial smoothing problems, which avoids smoothing across boundaries of a bounded non-convex spatial domain. 
  
  
  
  
  
  
  
  
  
  We observe data points {(x_i, y_i, z_i), i=1,…,n}, where z_i are the responses with random noise and {(x_i, y_i)} lie in a bounded region Ω⊂ℝ^2. The objective is to find the function f:Ω→ℝ which minimises
  
      ∑_i=1^n (z_i - f(x_i, y_i))^2 + λ∫_Ω( ∂^2f/∂ x^2 + ∂^2f/∂ y^2)^2 dx dy.
  
  The smoothing parameter λ is chosen through cross-validation. The soap film smoother is implemented in the R package  <cit.>.
  
  In our model (<ref>), the predicted Southern accent probabilities {p_ij} of individual speakers are observations at different locations {(_i, _i)} in Great Britain, and we use the soap film smoother to construct a smooth surface f(·, ·) to account for the geographic variation. We can compare the results using accent predictions from the two classification models proposed in the previous section.
  
  Plots of the fitted response surfaces μ̂(, ) = ( f̂(, ) ) using the formant and the MFCC classification models are given in Figure <ref>. 
  Both maps seem to suggest a North against Southeast split, similar to the isogloss map in Figure <ref>. The predicted probabilities are usually not close to 0 or 1, because the BNC contains more variation than we have in the NSCV training data, due for instance to the variation in recording environments, and since not all speakers have a stereotypical Northern or Southern accent. 
  
  
  
  
  
  
  
  
  
  To visualise the uncertainty associated with the contours in Figure <ref>, Figure <ref> shows the approximate 95% pointwise confidence intervals for μ. These are computed as [( f̂ - 1.96×(f̂)), (f̂ + 1.96×(f̂))], based on a normal approximation on the link function scale. Notice that the uncertainty for both models is high in Northern England, Scotland and Wales, due to fewer observations in those regions. However, the North-Southeast variation is consistent and Greater London emerges as a region with significantly Southern accents.
  
  
  
  
  
  
  
  § DISCUSSION
   
  
  We first demonstrated two principled and interpretable approaches to modelling accent variation in speech sounds, using techniques from functional data analysis and generalised additive models. We presented a model that uses formant trajectories to classify bath vowel sounds as Northern or Southern based on their similarity to /æ/ and /A/ vowels, trained on a set of labelled vowels[] collected in an experimental setup. The same audio dataset was also used in a different model using MFCC curves, by using functional principal components analysis to generate new features from the MFCC curves, and then classifying the sounds using ℓ_1-penalised logistic regression of the FPC scores. We showed in Section <ref> how this MFCC model allowed us to resynthesise vowel sounds along a spectrum between /æ/ and /A/.
  
  These formant and MFCC models were used to predict the probability of a Southern accent for vowels from the audio BNC <cit.>, our second dataset. The predictions were smoothed spatially to visualise the trap–bath split in England, Wales and Scotland, using a spatial beta regression with a soap film smoother. The resulting maps show a North versus South-east difference in accents which we can directly attribute to the variation in the /æ/ or /A/ vowel quality of BNC sounds.
  
  
  
  
  This analysis demonstrates how we can combine information from a labelled audio dataset such as the NSCV dataset, with the unlabelled BNC dataset. Despite the small sample of 4 speakers in the NSCV dataset, it allowed for vowel classification models to be trained. From cross-validation it seems that these classification models are highly accurate, a property that we believe would hold in similar recording conditions (such as background noise level) as the training data. 
  
  However, the classifiers can only distinguish between Northern and Southern BNC bath vowels to the extent that they differ by the /æ/ and /A/ vowels captured in the NSCV training dataset. To produce a more valid characterisation of accent variation, one could use a labelled dataset of speech recordings from a larger corpus of speakers who can produce both accents accurately. Another limitation of this analysis is that we cannot verify the assumption of smooth spatial accent variation since we have no accent labels for BNC sounds. An extension of this work could involve augmenting the BNC by having human listeners manually classify a random sample of BNC vowels as Northern or Southern. These labels could then be used to train accent classifiers directly on BNC vowels, and also to validate the assumption of smooth spatial accent variation.
  
  
  
  
  
  In phonetics, an ongoing research question has been whether dynamic information about formants is necessary for differentiating between vowels, or whether formant values at the midpoint of the vowel or sampled at fewer time points are sufficient <cit.>. In Appendix <ref> we have compared the functional formant model <ref> to simpler models using F_1 and F_2 formants measured at the middle of the vowel, or at 25%, 50% and 75% of the vowel. Even though the bath vowel is a monophthong which doesn't contain significant vowel transitions, we see that the functional models show slight increases in cross-validated classification accuracy as compared to their non-functional versions, and sampling formants at more time points does not hinder classification. This is due to the regularisation of smooth terms in the generalised additive models used. The functional modelling approach also doesn't require specific time points for sampling to be chosen in advance, so it can be easily used with other vowels which have different formant trajectories. The MFCC model is another higher dimensional approach to modelling variation in vowels, as it uses information from more frequency bands. It has a slightly lower accuracy than the functional formant model, but its ability to resynthesise vowel sounds may be desirable for some applications.
  
  
  
  
  
  We also observe some differences between the predictions of the formant model and the MFCC model. These models agree on vowel classification about 94% of the time for the NSCV vowels and 73% of the time for BNC vowels. The disagreements occur when the vowel quality is not clear, for example when formant curves are close to the boundary between the two vowels, or in the BNC, when there is considerable background noise and more variation in conversational speech. Nevertheless, the resulting spatial maps (Figure <ref> and Figure <ref>) show many similarities. Another way to compare the two models is to resynthesise vowels along a spectrum using the MFCC model, and classify these new vowels using the formant model. Changing the “Southernness” of a vowel with the MFCC model does change the corresponding prediction from the formant model, suggesting that similar information about vowel quality is being used by both models. We have added more detail on comparing the models in Appendix <ref>.
  
  It is also possible to combine both MFCCs and formants from the same sound in one model. This can be done similarly to the MFCC model <ref>, by appending the matrix of formant curves to the matrix of MFCC curves from each sound, and performing FPCA and ℓ_1-penalised logistic regression as before. The disadvantage of this model is that we can neither interpret it from a speech production perspective (since it contains MFCCs which don't have a physical interpretation), nor use it to resynthesise vowels (since we cannot resynthesise vowels using formants). We have nevertheless trained this model, which has an accuracy of 92.75%, and the results are in Appendix <ref>.
  
  
  
  The functional approach to modelling accent variation which we have demonstrated can easily be used with a larger corpus with more words or speakers. It can also be applied to to other vowels, including diphthongs (vowels which contain a transition, such as in “house”) to visualise other accent variation in Great Britain, or other geographic regions. 
  
  
  
  § SUPPLEMENTARY MATERIAL
  
  
  Data and R code: The R code and preprocessed data used to generate these results can be obtained [from the authors, on request]online at https://doi.org/10.5281/zenodo.4003815.
  
  Other outputs:  shows animated formant trajectories of the NSCV data. Resynthesised vowels are in  (perturbing the vowel in “class” from /æ/ towards /A/), and  (perturbing the vowel in “blast” from /A/ towards /æ/).
  
  
  
  § ACKNOWLEDGEMENTS
  
  
  We thank the Associate Editor and three referees for their comments that helped to improve the quality of the paper.
  
  
  
  
  agsm
  agsm
  
  
  
  
  
  
  
  
  
  
  
  § APPENDICES
  
  
  
  § EXPLORATION OF BNC
  
  
  
  
  Each observation in the dataset is a recording of a single utterance of a “class” word. The metadata that we have for each observation is shown in Table <ref>, along with the criteria used to clean the data. Since our interest is in the differences in natural native accents, we clean the dataset using the available metadata, to remove observations from trained speakers such as newsreaders, and speakers with foreign accents. We also remove observations from children, since the acoustic properties of their speech is considerably different from adult speech <cit.>. We limit our analysis to accents in Great Britain, and therefore removed recordings from Northern Ireland. Words that are shorter than 0.2 seconds or longer than 1 second were removed. 
  []We also removed the most noisy sounds using the procedure described in the online supplement of <cit.>.
  Histograms of the sound lengths in the BNC and NSCV are in Figure <ref>. 
  The cleaned dataset contains 3852 clips from 529 speakers in 124 locations across England, Scotland and Wales. Figure <ref> shows the number of observations per locations, Figure <ref> shows a histogram of utterances per speaker, and Table <ref> gives a breakdown of the social classes recorded in the metadata for the list of “class” words analysed.
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  § MODEL SELECTION FOR FORMANT MODEL
  
  
  
  Here are some descriptive statistics and plots of the NSCV formants, and the results from the model selection procedure. 
  
  
  
   §.§ Formants
   
  Table <ref> shows the average formant values for both vowels in the dataset, and Figure <ref> plots each sound's average formant values against each other. Figure <ref> shows the entire smoothed formant curves.
  
  
  
  
  
  
  
  
  
  
   §.§ Model selection
   
  
  Here are the results from the model selection procedure used to select the terms in the formant model. Table <ref> shows models that were trained on NSCV formants after aligning all NSCV MFCC 1 curves together, as described in Section <ref>. All models also had random effects for each speaker. We compare the effective degrees of freedom, AIC and cross validated accuracy for each model.
  
  The first three models use single formant values taken from the middle of the sound, models 4–6 use formant values taken at 3 points along the vowel (at 25%, 50% and 75% of the sound), and models 7–11 use the entire curve containing 40 samples from each curve. The models with formants at 3 time points are fitted in the same way as model <ref>, with the 3 formant values from each curve contributing to the predictor through a linear functional term, and the coefficient curves represented with a cubic spline with penalised second derivative.  We can see that the functional models usually have slightly higher accuracy and better fit to the data than their non-functional equivalents, and that increasing the number of samples from 3 to 40 doesn't meaningfully improve the classification accuracy.
  
  In the paper we use the functional model 8 in Table <ref> to classify vowels based on the F_2 curve, as it has the lowest AIC, second highest accuracy, and relatively low degrees of freedom.
  
  
  
  In Table <ref> we also show the same models trained on NSCV vowels before the alignment preprocessing step. For most of the models, the alignment has led to an increase in accuracy.
  In our procedure the alignment of vowels in the NSCV corpus is an important step as this allows us to align the BNC vowels to the NSCV vowels before predicting accents for the BNC. In general if misalignments between the training dataset and prediction dataset are not a concern, one can still obtain a good classifier using formants without the alignment step.
  
  
  
  
  
  
  
  
  § A MODEL COMBINING MFCCS AND FORMANTS
  
  
  
  Here is a model similar to model <ref>, in which we use both formant and MFCC curves together to classify vowels. In general applications usually use one form of speech representation at a time, as they would contain very similar information about the same sound. Following the same approach as the MFCC model, we first perform functional principal components analysis to reparametrise the 44 curves from each sound (40 MFCC curves and 4 formant curves), and use the resulting FPC scores in an ℓ_1-penalised logistic regression model.
  
  The cross validated accuracy of the model is 92.75% and 32 FPCs have nonzero coefficients. The confusion matrix is shown in Table <ref> and the spatial map of smoothed BNC predictions and the confidence intervals maps are shown in Figure <ref>. The overall spatial variation is very similar to what we see in maps from the separate formant and MFCC models.
  
  
  
  
  
  
  
  
  § COMPARING THE FORMANT-BASED MODEL AND MFCC-BASED MODEL
  
  
  
  The formant model <ref> and MFCC model <ref> sometimes classify the same vowel differently, which leads to some differences between the two spatial maps. In this Appendix we examine where these disagreements between the models occur.
  
  Figure <ref> shows the linear predictor values from the two models, when predicting the accent of NSCV vowels and BNC vowels. The models agree on the classification 94% of the time for NSCV vowels (when the vowels being predicted are not part of the training set), and 73% of the time for BNC vowels. The BNC predictions from both models on the linear predictor scale are closer to 0 than the NSCV predictions, because the BNC vowels are not as clearly differentiated as the vowels collected under controlled experimental conditions.
  
  
  
  We can also look at some features of the vowels themselves. Figure <ref> shows a sample of 50 F_2 curves from NSCV vowels where the models agree and all 25 curves where they disagree. The observations where the models disagree are not as clearly separated. A similar plot for BNC vowels is in Figure <ref>, coloured by their classifications under the MFCC model (since we don't know the ground truth).
  
  
  
  
  
  
  A third way of comparing the models is to resynthesise new vowels using the MFCC model, and classify the new vowels using the formant model. This allows us to compare one model's predicted probabilities against the other. 
  
  This is done similarly to the procedure in Section <ref>. We begin with a chosen vowel sound, preprocess its MFCCs, and predict its probability of being Southern under the MFCC model. Using this prediction on the linear predictor scale, we can calculate the sequence of multiples of the MFCC matrix (Figure <ref>) to add to the sound in order to achieve certain predictions under the MFCC model. We can add these MFCCs to the original MFCCs and synthesising the resulting sound to obtain a new sequence of vowels. We then extract and preprocess the formants from these sounds, and calculate their predictions under the formant model. Examples of this for two Northern and two Southern vowels are shown in Figure <ref> where we perturb vowels between -12 and 12 on the MFCC linear predictor scale. Figure <ref> shows the same plot on the probability scale.
  
  This shows us that the formant model validates the vowel change that we can create using the MFCC model. It suggests that the sound information being used by both these models is closely related.
  
  
  
  
  
  