abstract: This paper presents an efficient parallel Adaptive Inverse Distance Weighting (AIDW) interpolation algorithm on modern Graphics Processing Unit (GPU). The presented algorithm is an improvement of our previous GPU-accelerated AIDW algorithm by adopting fast $k$-**N**earest **N**eighbors ($k$NN) search. In AIDW, it needs to find several nearest neighboring data points for each interpolated point to adaptively determine the power parameter; and then the desired prediction value of the interpolated point is obtained by weighted interpolating using the power parameter. In this work, we develop a fast $k$NN search approach based on the space-partitioning data structure, even grid, to improve the previous GPU-accelerated AIDW algorithm. The improved algorithm is composed of the stages of $k$NN search and weighted interpolating. To evaluate the performance of the improved algorithm, we perform five groups of experimental tests. Experimental results show that: (1) the improved algorithm can achieve a speedup of up to 1017 over the corresponding serial algorithm; (2) the improved algorithm is at least two times faster than our previous GPU-accelerated AIDW algorithm; and (3) the utilization of fast $k$NN search can significantly improve the computational efficiency of the entire GPU-accelerated AIDW algorithm.
address: School of Engineering and Technology, China University of Geosciences, Beijing 100083, China
author: Gang Mei; Nengxiong Xu; Liangliang Xu
bibliography: MG.bib
title: Improving GPU-accelerated Adaptive IDW Interpolation Algorithm
  Using Fast *k*NN Search

# Introduction

Spatial interpolation is a fundamental tool in Geographic Information System (GIS). The most frequently used spatial interpolation algorithms include the Inverse Distance Weighting (IDW) , Kriging , Discrete Smoothing Interpolation (DSI) , nearest neighbors, etc; see a comparative survey investigated by . When applying those interpolation algorithms for large-scale datasets, the computational cost is in general too high . A common and effective solution to the above problem is to perform the interpolation procedures in parallel. Currently, many efforts have been carried out to parallelize those interpolation algorithms in various environments on multi-core CPU and/or many-core GPU platforms .

For example, to accelerate the Kriging method, proposed a solution to parallelizing the ordinary Kriging using MPI (Message Passing Interface) libraries in an HPC (High Performance Computing) environment, and significantly reduced the final execution time of the entire process. Similarly, presented a new parallel Kriging algorithm to deal with unevenly spaced data. proposed an efficient parallel scheme to accelerate the universal Kriging algorithm on the NVIDIA CUDA platform by optimizing the compute-intensive steps in the Kriging algorithm, such as matrix–vector multiplication and matrix–matrix multiplication and achieved a nearly 18 speedup over the serial program.

introduced an efficient out-of-core algorithm that fully benefited from graphics cards acceleration on a desktop computer, and found that it was able to speed up Kriging on the GPU with data 4 times larger than a classical in-core GPU algorithm, with a limited loss of performances.

To improve the computational efficiency of the most time-consuming steps in ordinary Kriging, i.e., the weights calculation and then the estimate for each unknown point, investigated the potential reduction in execution time by selecting the suitable operations involved in those steps to be parallelized by using general-purpose computing on GPUs and CUDA.

proposed an improved coarse-grained parallel algorithm to accelerate ordinary Kriging interpolation in a homogeneously distributed memory system using the MPI (Message Passing Interface) model and achieved the speedups of up to 20.8. proposed an algorithm based on the $k$-d tree method to partition a big dataset into workload-balanced child data groups, and achieved high efficiency when the datasets were divided into an optimal number of child data groups.

The IDW interpolation algorithm has been also parallelized on various platforms. For example, exemplified by a hybrid IDW algorithm to generate DEM from LiDAR point clouds, designed and implemented a parallel pipeline algorithm for multi-core platforms, and processed nearly one billion LiDAR points in about 12 min and produced a 27,500 $\times$ 30,500 raster DEM using less than 800 MB of main memory on a 2.0 GHz Quad-Core Intel Xeon platform. accelerated the IDW method on the GPU for predicting the snow cover depth at the desired point.

proposed a generic methodological framework for geospatial analysis based on GPU and explored how to map the inherent parallelism degrees of IDW interpolation, which gave rise to a high computational throughput. explored of the implementation of a parallel IDW interpolation algorithm in a Linux cluster-based parallel GIS. developed their IDW interpolation application uses the Java Virtual Machine (JVM) for the multi-threading functionality.

developed two GPU implementations of the standard IDW interpolation algorithm, the tiled version and the CDP version, by exploiting the shared memory and CUDA Dynamic Parallelism, and observed that the tiled version can achieve the speedups of 120 and 670 over the CPU version when the power parameter was set to 2 and 3.0, respectively. also evaluated the impact of several data layouts on the efficiency of GPU-accelerated IDW interpolation.

Some of the other efforts have been also carried out to parallelize other interpolation algorithms. For example, presented a computing scheme to speed up the Projection-Onto-Convex-Sets (POCS) interpolation for 3D irregular seismic data with GPUs. developed a parallel the fast Fourier transform (FFT) based geostatistical areal interpolation algorithm in a homogeneously distributed memory system using the MPI programming model. employed the $k$-d tree in nearest neighbors search to accelerate the grid interpolation on the GPU. proposed a parallel method based on radial basis functions for surface reconstruction on GPU.

The Adaptive IDW (AIDW) is an improved version of the standard IDW , which was originally proposed by . The basic and most interesting idea behind the AIDW is that: it attempts to calculate the power parameter adaptively according to the spatial distribution pattern of the data points, while in the standard IDW the power parameter is a user-specified constant value. Due to the adaptive determination of the power parameter, the AIDW method can achieve much more accurate prediction results than those by the standard IDW.

In our previous work , we have designed and implemented a parallel AIDW algorithm on a GPU. And we have also evaluated the performance of the parallel AIDW method by comparing its efficiency with that of the corresponding serial one. We have observed that our GPU-accelerated AIDW algorithm can achieve the speedups of up to 400 for one million data points and interpolated points on single precision.

In our previous GPU implementations of the parallel AIDW method, we have found that the most computationally intensive step is the $k$ Nearest Neighbors ($k$NN) search for each interpolated points. We have designed a straightforward method to find the $k$ nearest neighboring data points for each interpolated point within a single thread. Although the GPU implementing using our straightforward $k$NN search approach can achieve satisfied computational efficiency, for example, the obtained speedups are about 100 $\sim$ 400 on single precision, further performance improvement probably can be achieved by optimizing the $k$NN search.

The task of the $k$NN search is to find the nearest neighbors to an input query. Previous research works on the $k$NN search are mainly implemented and optimized in CPU . Recently, GPU-accelerated implementations have improved performance by utilizing the massively parallel architecture of a single GPU , multi-GPUs , and GPU clusters . Among those GPU-accelerated $k$NN search algorithms, most of them focusing on speeding up the brute-force $k$NN search algorithm; and several of them are designed and optimized using space partitioning data structures such as grid , RP-tree , VP-tree , and $k$-d tree .

In this paper, we attempt to improve the efficiency of our previous GPU-accelerated AIDW algorithm by adopting a more efficient $k$NN search approach. The efficient $k$NN search is expected to be performed in a separate stage with the use of the data structure, grid. The resulting values of the $k$NN search are the distances between the $k$ nearest neighboring data points to each interpolated point. Those distances are then transferred into another stage of the AIDW to adaptively calculate the power parameter and the expected prediction value (i.e., the weighted average). To evaluate the improved parallel AIDW algorithm, we also compare its efficiency with that of our previous one introduced in .

The rest of this paper is organized as follows. Section 2 introduces the background principles of the IDW algorithm, the AIDW algorithm, and the $k$NN search. Section 3 describes the strategies and considerations for improving our previous GPU-accelerated AIDW algorithm. Section 4 presents some implementation details of the improved algorithm. Some comparative experimental tests and analysis are provided in Section 5. Finally, Section 6 draws several conclusions.

# Background

This section will briefly introduce the principles of the standard IDW interpolation method , the AIDW interpolation method , and the $k$NN search.

## The Standard IDW Interpolation

The IDW algorithm is one of the most popular spatial interpolation methods in Geosciences, which calculates the prediction values of unknown/interpolated points by weighting average of the values of known/data points. The name given to this type of methods was motivated by the weighted average applied since it resorts to the inverse of the distance to each known point when calculating the weights. The difference between different forms of IDW interpolation is that they calculate the weights variously.

A general form of predicting an interpolated value $Z$ at a given point $x$ based on samples $Z_{i}=Z(x_{i})$ for $i$ = 1, 2, …, $n$ using IDW is an interpolating function:

$$\label{eq1}
    Z(x)=\sum\limits_{i=1}^n {\frac{\omega _i (x)z_i }{\sum\limits_{j=1}^n
            {\omega _j (x)} }} ,
    \quad
    \omega _i (x)=\frac{1}{d(x,x_i )^\alpha }.$$

The above equation is a simple IDW weighting function, as defined by , where $x$ denotes a prediction location, $x_{i }$ is a data point, $d$ is the distance from the known data point $x_{i}$ to the unknown interpolated point $x$, $n$ is the total number of data points used in interpolating, and $p$ is an arbitrary positive real number called the power parameter or the distance-decay parameter (typically, $\alpha$ = 2 in the standard IDW). Note that in the standard IDW, the power/distance-decay parameter $\alpha$ is a user-specified constant value for all unknown interpolated points.

## The AIDW Interpolation

The AIDW is an improved version of the standard IDW , which is originated by . The basic and most interesting idea behind the AIDW is that: it adaptively determines the distance-decay parameter $\alpha$ according to the spatial pattern of data points in the neighborhood of the interpolated points. In other words, the distance-decay parameter $\alpha$ is no longer a pre-specified constant value but adaptively adjusted for a specific unknown interpolated point according to the distribution of the nearest neighboring data points.

When predicting the desired values for the interpolated points using AIDW, there are typically two phases: the first one is to determine adaptively the power parameter $\alpha$ according to the spatial pattern of data points; and the second is to perform the weighting average of the values of data points. The second phase is the same as that in the standard IDW; see Equation ().

In AIDW, for each interpolated point, the parameter $\alpha$ can be adaptively determined according to the following steps.

**Step 1**: Determine the spatial pattern by comparing the observed average nearest neighbor distance with the expected nearest neighbor distance.

1.  Calculate the expected nearest neighbor distance $r_{\exp }$ for a random pattern using: $$\label{eq2}
        r_{\exp } =\frac{1}{2\sqrt {n \mathord{\left/ {\vphantom {n A}} \right.
                    \kern-\nulldelimiterspace} A} },$$ where $n$ is the number of points in the study area, and $A$ is the area of the study region.

2.  Calculate the observed average nearest neighbor distance $r_{obs}$ by taking the average of the nearest neighbor distances for all points:

    $$\label{eq3}
        r_{obs} =\frac{1}{k}\sum\limits_{i=1}^k {d_i } ,$$ where $k$ is the number of nearest neighbor points, and $d_i$ is the nearest neighbor distances. The $k$ can be specified before interpolating.

3.  Obtain the nearest neighbor statistic $R\left( {S_0 } \right)$ by:

    $$\label{eq4}
        R\left( {S_0 } \right)=\frac{r_{obs} }{r_{\exp } },$$ where $S_{0 }$ is the location of an interpolated point.

**Step 2**: Normalize the $R\left( {S_0 } \right)$ measure to $\mu _R$ such that $\mu _R$ is bounded by 0 and 1 by a fuzzy membership function: $$\label{eq5}
\mu _R =\left\{ {\begin{array}{ll}
    0&R\left( {S_0 } \right)\le \mbox{ }R_{\min } \mbox{ } \\
    0.5-0.5\cos \left[ {\frac{\pi }{R_{\max } }\left( {R\left( {S_0 }
            \right)-R_{\min } } \right)} \right]&R_{\min } \le R\left( {S_0 }
    \right)\le \mbox{ }R_{\max } \mbox{ } \\
    1&R\left( {S_0 } \right)\ge \mbox{ }R_{\max } \\
    \end{array}} \right.,$$ where $R_{\min } \mbox{ }$ or $R_{\max }$ refers to a local nearest neighbor statistic value (in general, the $R_{\min } \mbox{ }$ and $R_{\max }$ can be set to 0.0 and 2.0, respectively).

**Step 3**: Determine the distance-decay parameter $\alpha$ by mapping the $\mu _{R}$ value to a range of $\alpha _{ }$ by a triangular membership function that belongs to certain levels or categories of distance-decay value; see Equation (). $$\label{eq6}
\alpha \left( {\mu _R } \right)=\left\{ {{\begin{array}{ll}
        {\alpha _1 } & {\mbox{0.0}\le \mu _R \le \mbox{0.1}} \\
        {\alpha _1 \left[ {1-5\left( {\mu _R -0\mbox{.}1} \right)} \right]+5\alpha
            _2 \left( {\mu _R -0\mbox{.}1} \right)} & {\mbox{0.1}\le \mu _R \le
            \mbox{0.3}} \\
        {5\alpha _3 \left( {\mu _R -0\mbox{.}3} \right)+\alpha _2 \left[ {1-5\left(
                {\mu _R -0\mbox{.}3} \right)} \right]} & {\mbox{0.3}\le \mu _R \le
            0\mbox{.}5} \\
        {\alpha _3 \left[ {1-5\left( {\mu _R -0\mbox{.5}} \right)} \right]+5\alpha
            _4 \left( {\mu _R -0\mbox{.}5} \right)} & {\mbox{0.5}\le \mu _R \le
            \mbox{0.7}} \\
        {5\alpha _5 \left( {\mu _R -0\mbox{.7}} \right)+\alpha _4 \left[ {1-5\left(
                {\mu _R -0\mbox{.7}} \right)} \right]} & {\mbox{0.7}\le \mu _R \le
            \mbox{0.9}} \\
        {\alpha _5 } & {\mbox{0.9}\le \mu _R \le \mbox{1.0}} \\
        \end{array} }} \right.,$$ where the $\alpha _{1}$, $\alpha _{2}$, $\alpha _{3}$, $\alpha
            _{4}$, $\alpha _{5}$ are the assigned to be five levels or categories of distance-decay value.

After determining the parameter $\alpha$, the desired prediction value of each interpolated point can be obtained via the weighting average. This stage is the same as that in the standard IDW; see Equation ().

## $k$NN Search

The principle and major steps of the brute-force $k$NN search are as follows :

Considering a set $R$ of $m$ reference points in a *d*-dimensional space $R=\{r_1 ,r_2 ,.....,r_m \}$, and a set $Q$ of $n$ query points in the same space $Q = \mbox{\{}q_1 ,q_2 , ... , q_n \mbox{\}}$, for a query point $q \in Q$, the brute-force algorithm is composed of the following steps:

1\) Compute the distance between $q$ and the $m$ reference points of $R$:

2\) Sort the $m$ distances;

3\) Output the distances in the ordered of increasing distance.

When applying this algorithm for the $n$ query points with considering the typical case of large sets, the complexity of this algorithm is overwhelming:

-   $O(nmd)$ multiplications for the $n\times m$ distances computed;

-   $O(nm\log m)$ is for the $n$ sorting processes.

The brute-force $k$NN search method is by nature highly parallelizable and perfectly suitable for a GPU implementation.

# The Improved GPU-accelerated AIDW Method

This section will briefly introduce the considerations and strategies in the development of the improved GPU-accelerated AIDW interpolation algorithm.

## Overview and Basic Ideas

The basic and most interesting concept behind the AIDW method is that: it attempts to determine adaptively the power parameter according to the spatial distribution pattern of each interpolated point. In AIDW algorithm, the spatial distribution pattern is considered as the distribution density of several nearest neighboring data points locating around an interpolated point, which can be roughly measured by using the average distance from those neighboring data points to the interpolated point.

In our previous work, we present a straightforward, easy-to-implement, and suitable for GPU-parallelization algorithm to find the $k$ nearest neighboring data points of each interpolated point. Assuming there are $n$ interpolated points and $m$ data points, for each interpolated point we carry out the following steps :

Step 1: Calculate the first $k$ distances between the first $k$ data points and the interpolated points;

Step 2: Sort the first $k$ distances in ascending order;

Step 3: For each of the rest ($m-k)$ data points,

1\) Calculate the distance *dist*;

2\) Compare the *dist* with the $k$th distance:

if *dist* $<$ the $k$th distance, then replace the $k$th distance with the *dist*

3\) Iteratively compare and swap the neighboring two distances from the $k$th distance to the first distance until all the $k$ distances are newly sorted in ascending order.

The major advantage of the above algorithm is that: it is simple and easy to implement. Obviously, there is no need to utilize any complex space partitioning data structures such as various types of *trees*. In contrast, only arrays for storing distances and coordinates are needed. Also, we find the desired nearest neighbors without the use of explicit sorting algorithms such as binary search. In general, most sorting algorithms are computationally complex and not suitable for entirely being invoked within a single GPU thread.

The most obvious shortcoming of the above algorithm for finding nearest neighboring data points is that: it is computationally inefficient due to the global search for nearest neighbors. In that algorithm, the first $k$ distances are calculated and recorded; and then the distances to the rest points are calculated and then compared with those first $k$ distances. The above procedure obviously needs a global search, which is not computationally optimal. One of the frequently used optimization strategies is to perform a local search by filtering those data points and distances that are not needed to be considered.

In this work, we focus on improving our previous GPU-accelerated AIDW algorithm by using a fast $k$NN search algorithm. Our considerations and basic ideas behind developing the efficient $k$NN search algorithm are as follows:

\(1\) Create an even grid to partition the planar region that encloses the projected positions of all data points and interpolated points;

\(2\) Distribute all the data points and interpolated points into the grid and record the locations;

\(3\) Perform a *local* and fast search within the grid to find the nearest neighboring data points for each interpolated point.

After obtaining the average distance of those neighboring data points, the adaptive power parameter $\alpha$ will be determined according to the average distance. Finally, the desired prediction value for each interpolated point can be obtained via weighting average using the parameter $\alpha$; see more descriptions in Subsection 2.2.

In summary, the improved GPU-accelerated AIDW algorithm is mainly composed of two stages: (1) the $k$NN search and average distances calculation, and (2) the determination of adaptive power parameter and prediction value by weighted interpolating; see Figure .

## Stage 1: $k$NN Search

The workflow of the stage of $k$NN search is listed in Figure . In this section, more descriptions on this stage will be presented.

### Creating an Even Grid

The even grid is a simple type of data structure for space partitioning, which is composed of regular cells such as squares or cubes; see an example of planar grid illustrated in Figure . Compared to other efficient but complex space partitioning data structures such as the $k$-d tree, the even grid is much easier to create and search objects. In this work, we use a planar even grid to partition all data points to speed up the $k$NN search via local search.

The building of an even planar grid is straightforward. We first calculate or specify the width of the square cell, then determine the planar rectangular region for partitioning according to the minimum and maximum $x$ and $y$ coordinates of all points, i.e., obtain the length and width of the rectangle. After that, the numbers of rows and columns of the grid can be quite easily determined by dividing the rectangle.

### Distributing Data Points into Cells

The distribution of each data point is to find out that in which grid cell the data point locates. Since each grid cell can be located and recorded using its row and column indices, the distribution of each data point is in fact to obtained the row and column indices of the cell in which it locates.

This procedure can also be quite easily performed. First, the differences between the coordinates of the data points and the minimum coordinates of all cells are calculated; then the indices of row and column can be obtained by dividing the above differences with the cell width.

### Determining Data Points in Each Cell

The most important and basic idea behind utilizing a space partitioning is to perform a local search within local regions rather than a global search. When searching nearest neighbors, it is computationally optimal to first search approximate nearest neighbors within several local cells and then to find the exact nearest neighbors by filtering undesired points.

Since the local search is operated within cells, it is thus needed to determine that which data points locate inside a specific cell. In other words, it is needed to know the number and the indices of those data points locating in the same cell. Moreover, the layout for storing the number and indices should be carefully handled.

For each grid cell, to store the above-mentioned number and indices of those data points locating in the same cell, in general, a dynamic array of integers needs to be allocated. In the traditional CPU computing, the allocation and operations of dynamic arrays are easy-to-implement and computationally inexpensive. However, in GPU computing, it is no longer easy to implement or computationally cheap. This is because that: (1) in GPU computing the programming model such as CUDA cannot support the allocation and operations of dynamic arrays/containers like `vector` and `list` in C++ STL (Standard Template Library); and (2) the allocation of a large-enough static array of integers, e.g., `int index[1000]`, for storing the indices of data points within each GPU thread is not memory efficient.

Due to the above reasons, we design an optimal layout for storing the number and indices of data points. The basic idea is that: if the indices of those points locating inside the same cell are stored in a continuous segment/piece of integer values, then we only need to know the address of the first point in the segment and the number of points in the same segment (i.e., the size of the segment).

In this case, for each cell, we can only use two integer values to record the number and the indices of those data points that locate in the same cell. One integer is used to hold the number, and the other is used to record the address of the head/first point in each segment. The above two values can be very efficiently determined in a parallel fashion.

Before determining the number and indices of data points locating in the same cell, those data points should be recorded continuously. Since we have obtained the index of the cell in which each data point locates, if we sort all data points according to their corresponding cell indices in ascending order, then those data points locating in the same cell can be gathered in a continuous segment. This sorting procedure is suited to be parallelized on the GPU.

The number of data points locating in the same cell is determined using *segmented* parallel reduction. As described above, after sorting all data points according to cell indices, all data points are stored in a group of segments; each segment is flagged with the cell index, and contains the indices of data points locating in the same cell. The number of data points locating in the same cell can be achieved by performing a reduction for each segment; see Figure (a). Similarly, the head index of the first point of each segment can be obtained using segmented parallel scan; see Figure (b).

### Searching Nearest Neighbors

In this work, a space-partitioning data structure, the even grid, is employed to enhance the $k$NN search algorithm. The most important and basic idea behind utilizing the space partitioning is to perform a local search within local regions rather than a global search. This idea is quite effective in practice for that the number of points that are needed to find and compare can be significantly reduced, and therefore, the computational efficiency can be improved.

The process of $k$NN search for each interpolated point can be summarized as follows.

-   Step 1: Locate the interpolate point into the even grid

-   Step 2: Determine the level of cell expanding

-   Step 3: Find the nearest neighbors within the local region

-   Step 4: Calculate the average distance

The locating of each interpolated point into the previously created planar grid is quite straightforward. Since each grid cell can be located and recorded using its row and column indices, the distribution of each interpolated point is in fact to obtained the row and column indices of the cell in which it locates. First, the differences between the coordinates of the interpolated point and the minimum coordinates of all cells are calculated; then the indices of row and column can be obtained by dividing the above differences with the cell width.

The determining of the level of cell expanding is in fact to determine the region of cells in which the local nearest neighbors search should be carried out; see three levels of cell expanding in Figure . In $k$NN search, the number of nearest neighbors, $k$, is typically pre-specified; and obviously, the number of data points locating in the local cells must be larger than the number $k$. Thus, the level of cell expanding can be iteratively determined by comparing the number of currently found data points with the number $k$. For example, when the $k$ is specified as 15, and within the first level of local cells there are only 10 data points, and thus the level 1 needs to expand to level 2. Similarly, if only 14 data points can be found within the second level of local cells, the level needs to be further expanded to 3. This procedure is iteratively repeated until enough data points have been found.

**Remark:** Note that after iteratively determining the level of cell expanding, for example, level 3, the final level of cell expanding needs to increase with 1, i.e., level 4. This is because that: without expanding additional one level, the nearest neighbors found in the initial level of local cells may not the desired exact $k$ nearest neighbors; see the marked data point in Figure . When $k$ = 10, the determined level of cell expanding is 0 (i.e., the yellow region). However, the marked data point is obvious one of the nearest neighbors of the only interpolated point because it is much nearer to the interpolated point than several data points locating in the yellow region. This demonstrates that: without expanding additional one level, incorrect/undesired nearest neighboring data points are probably found; and several of the expected nearest neighboring data points may not able to be found.

The $k$NN search in the local cells is, in fact, to further find exact nearest neighbors by filtering some undesired points. We first allocate an array with the size of $k$ for storing distances, and initiate all distances to 0. Then for each of those data points locating in the local cells, we calculate the distance *dist*, and compare the *dist* with the $k$th distance; and if *dist* is smaller than the $k$th distance, then replace the $k$th distance with the *dist*; after that, we iteratively compare and swap the neighboring two distances from the $k$th distance to the first distance until all the $k$ distances are newly sorted in ascending order; see for more details.

After finding the nearest neighbors of each interpolated point, the distances between each nearest neighbor and the interpolated point can be calculated; and finally, the desired average distance can be obtained.

## Stage 2: Weighted Interpolating

Due to the inherent feature of the AIDW interpolation algorithm, it is perfect that a single GPU thread can take the responsibility to calculate the prediction value of an interpolated point. For example, assuming there are $n$ interpolation points that are needed to be predicted their values such as elevations, and then it is required to allocate $n$ threads to calculate the desired prediction values for all those $n$ interpolated points concurrently.

In GPU computing, shared memory is inherently much faster than global memory; thus, any opportunity to replace global memory access by shared memory access should therefore be utilized. Since the shared memory residing in the GPU is limited per SM (Stream Multiprocessor), a common optimization strategy called "tiling" is frequetnly used to handle the above problem, which partitions the data stored in global memory into subsets called tiles so that each tile fits into the shared memory.

This optimization strategy "tiling" is also adopted to accelerate the AIDW interpolation algorithm: the coordinates of all data points are first transferred from global memory to shared memory; then each thread within a thread block can access the coordinates stored in shared memory concurrently. By exploiting the "tiling" strategy, the global memory access can be significantly reduced; and thus, the overall computational efficiency is expected to be improved.

# Implementation Details

As introduced in the above section, the improved GPU-accelerated AIDW interpolation algorithm is mainly composed of two stages, i.e., the $k$NN search stage and the weighted interpolating stage. In this section, we will describe some implementation details on the above two stages.

## Stage 1: $k$NN Search

### Creating an Even Grid

An even grid is composed of a group of grid cells, and in this work, each grid cell is a square. The creation of an even grid is in fact to determine the position of the grid, the size of the cell, and the distribution layout of the cells. In our algorithm, an even planar grid is created to cover the planar region in which the projected positions of all data points and interpolated points locate.

We first obtain the minimum and maximum coordinates of all the data points and interpolated points using the parallel reduction `thrust::minmax``_``element()` provided by the library *Thrust* , and calculate the differences between those minimum and maximum coordinates in $x$- and $y$- direction. After approximately determining the planar region, we then calculate the length of interval `cellWidth`, i.e., the width of a square cell, according to Equation (). After that, the number of rows and columns of grid cells can be easily calculated as follows:

`int nCol = (maxX - minX + cellWidth) / cellWidth;`

`int nRow = (maxY - minY + cellWidth) / cellWidth;`

### Distributing Data Points into Cells

After creating the even grid, the subsequent step is to distribute all the data points into the grid. This procedure can be naturally parallelized since the distributing of each data point can be performed independently. Assuming there are $m$ data points, we allocate $m$ GPU threads to distribute all the data points. Each thread is responsible for calculating the position of one data point locating in the grid, i.e., to determine the index of the cell where the data point locates. This can be very easily achieved using the following formulations.

`int col``_``idx = (int) (dx[i] - minX) / cellWidth;`

`int row``_``idx = (int) (dy[i] - minY) / cellWidth;`

A cell in a grid can be exactly positioned according to the indices of row and column, i.e., `int col``_``idx`, `row``_``idx`. Also, the position of each grid can be found according to its global index that can be calculated using the simple transformation, `global``_``idx = row``_``idx * nCol + col``_``idx`.

The above transformation formulation can be used to transform a two-dimensional index of each grid cell to a unique one-dimensional index. Obviously, this transformation can be easily transformed back. The reason why we carry out the transformation is that: first the memory requirement is reduced since only one array of integers is needed to be stored, and the second is that sorting with using one value as the key is much faster than that with two values as keys.

To obtain the indices and numbers of those data points locating in each cell, an effective solution is to store those data points that locate in the same cell continuously. Then, operations on the continuous pieces of data (i.e., segments) can be very efficient; see more descriptions in the closely subsequent section.

### Determining Data Points in Each Cell

In the stage of the $k$NN search, our objective is to find $k$ nearest neighboring data points for each interpolated point. The $k$NN search for each interpolated point is locally performed within several grid cells. The first requirement is to determine how many and which data points locate in each grid cell. More specifically, we need to know the indices and the number of those data points locating in each grid cell. We obtain this simply by using parallel reduction and scan; see our ideas illustrated in Figure .

Before carrying out the parallel reduction and scan, those data points that locate inside the same cell should be stored continuously. This requirement can be fulfilled by utilizing a parallel sort with the use of the global index of cells as keys. The parallel sort is realized by using the corresponding parallel primitive provided by the powerful library *Thrust*, `thrust::sort``_``by``_``key(keys, values)`.

Note that those data points locating in the same cell are stored continuously, and if we know the number of data points locating in the same cell, then we only to know the first address of the first data point; and each of the rest data points can be referenced according to the address of the first point and its local position. This idea is quite similar to the reference of any value/element in an array.

Then, the parallel reduction and scan are also performed by using the primitives provided by *Thrust*. We also use the global index of cells as the keys for ***Segmented*** reduction and scan. The motivation why we use the segmented reduction and scan rather than the global reduction and scan is that: in the current step we only need to operate on the data points locating in the same cell; and those data points locating in the same cell have been stored continuously and marked using the global index of cell as flags; see Figure .

The number of those data points locating in the same cell is obtained by using the primitive `thrust::reduce``_``by``_``keys()`; and the index of the first/head point of each segment of data points are found using `thrust::unique``_``by``_``keys()`. As illustrated in Figure , a helper array of constant integers is additionally used to count the number of data points stored in the same piece/segment.

### Searching Nearest Neighbors

The finding of $k$ nearest neighboring data points for each interpolated points can be inherently parallelized. Assuming there are $n$ interpolated points, and we allocate $n$ threads to search the nearest neighbors for all the interpolated points. Each thread is invoked to find the nearest neighbors for only one interpolated point.

Within each thread, we first distribute the interpolated point into the created grid by calculating its row index and column index; see lines 13 $\sim$ 14 in Figure . Then we determine the region of the local cells by approximately calculating the level of expanding according to the number of data points; see lines 16 $\sim$ 29 in Figure . Note that currently those data points locating in the determined local cells are the ***Approximate*** nearest neighbors of the interpolated points. After that, we further find the ***Exact*** nearest neighbors by filtering those approximate nearest neighbors by inserting and swapping; see lines 31 $\sim$ 58 in Figure . Finally, the desired average distance between the exact nearest neighboring data points and the target interpolated point is calculated.

A remarkable implementation detail is that: when finding the nearest neighbors according to the *Euclidean* distances between points, we do not use the real distance value but the square value of the distance. This is because that: in GPU computing the calculation of square root is quite computationally expensive; and any choice to avoid the use of calculating square root should be exploited. Thus, we calculate the square root in the last step of computing the average distance, rather in the step of searching nearest neighbors.

## Stage 2: Weighted Interpolating

This subsection will present the details on implementing the interpolating stage in the GPU-accelerated AIDW algorithm. We implement two versions: the *naive* version and the *tiled* version, by employing the data layout Structure-of-Arrays (SoA) only. Both the naive and the tiled implementations developed in this work are the same as those corresponding implementations presented in our previous work .

### Naive Version

In this version, the global memory and registers on GPU architecture are employed without exploiting the shared memory. The input data and the output data are stored in the global memory. Assuming that there are $m$ data points used to evaluate the prediction values for $n$ interpolation points, we allocate $n$ threads to parallelize the interpolating.

The data layout SoA is employed in this version. The coordinates of all data points and interpolated points are stored in the arrays `float dx[dnum]`, `dy[dnum]`, `dz[dnum]`, `ix[inum]`, `iy[inum]`, and `iz[inum]`.

Since that after invoking the $k$NN kernel, we have obtained the average distance, i.e., the $r_{obs}$ defined in Equation (), thus in this stage each thread is only responsible for computing the $r_{\exp }$ and $R\left(
            {S_0 } \right)$ according to the Equations () and (). After that, the $R\left( {S_0 } \right)$ measure is normalized to $\mu _R$ such that $\mu
            _R$ is bounded by 0 and 1 by a fuzzy membership function; see Equation (). Finally, the power parameter $\alpha$ is determined by mapping the $\mu
            _{R}$ values to a range of $\alpha _{ }$ by a triangular membership function; see Equation ().

After adaptively determining the power parameter, the desired prediction value of each interpolated point can be achieved by weighting average. This step of calculating the weighting average is the same as that in the standard IDW method.

### Tiled Version

The workflow of the tiled version is the same as that of the naive version. The major difference between the two versions is that: in this version, the shared memory is exploited to improve the computational efficiency.

In the tiled version, the tile size is directly set to be identical to the block size. Each thread within a thread block is invoked to load the coordinates of one data point from global memory to shared memory and then compute the distances and corresponding inverse weights to those data points stored in current shared memory. After all threads within a block finished computing these partial distances and weights, the next piece of data in global memory is loaded into shared memory and used to calculate current wave of partial distances and weights. After calculating each wave of partial distances and weights, each thread accumulates the results of all partial weights and all weighted values into two registers. Finally, the prediction value of each interpolated point can be obtained according to the sums of all partial weights and weighted values and then written into global memory.

By employing the strategy "tiling", the global memory access can be significantly reduced for that the coordinates of all data points are only read ($n$/ threadsPerBlock) times rather than $n$ times from global memory, where $n$ is the number of interpolated points and threadsPerBlock denotes the number of threads per block.

# Results and Discussion

## Experimental Environment and Testing Data

In this work, we focus on improving our previous GPU-accelerated AIDW algorithm by utilizing a fast $k$NN search method. We refer our previously developed GPU-accelerated AIDW algorithm as the *original* algorithm, and the presented algorithm in this work as the *improved* algorithm.

To evaluate the computational efficiency of the improved algorithm, we have carried out five groups of experimental tests on a laptop computer. The computer is featured with an Intel Core i7 CPU (2.40GHz), 4.0 GB RAM memory, and a GeForce GT730M card. All the experimental tests are executed on OS Windows 7 Professional (64-bit), Visual Studio 2010, and CUDA v7.0.

Two versions of the improved GPU-accelerated AIDW, i.e., the naive version and the tiled version, are implemented using the SoA layout and evaluated on single precision. In contrast, the CPU version of the AIDW implementation is tested on double precision; and all results of this CPU version presented in our previous work are directly accepted to be used as the baseline. The efficiency of all GPU implementations is benchmarked by comparing to the baseline results.

When evaluating the execution time of GPU implementations, the overhead spent on transferring the input data (i.e., the coordinates of data points and interpolated points) from the host side to the device side and transferring the results from the device side to the host side is considered. However, the time spent on generating the test data is not included.

The input of the AIDW interpolation is the coordinates of data points and interpolated points. The efficiency of the CPU and GPU implementations may differ due to different sizes of input data. However, the research objective in this work is to improve our previous GPU-accelerated AIDW algorithm using fast $k$NN search; thus, we only consider a particular situation where the numbers of interpolated points and data points are identical.

All the testing data including the data points and interpolated points are randomly generated within a square. We design five groups of sizes, i.e., 10K, 50K, 100K, 500K, and 1000K, where one K represents the number of 1024 (1K = 1024). Five tests are performed by setting the numbers of both the data points and interpolated points as the above five groups of sizes.

## Performance of the Improved GPU-accelerated AIDW Algorithm

### Executing Time and Speedups

We evaluate the computational efficiency of the improved GPU-accelerated AIDW algorithm with the use of five groups of testing data. The running time is listed in Table . Note that, to compare with the original GPU-accelerated algorithm, we have also listed the execution time of the original algorithm in Table ; and these experimental results of the original algorithm are directly derived from our previous work .

| \[0cm\]\[0cm\]Version  | Data Size (1K = 1024) |        |        |          |          |
|:-----------------------|:----------------------|:-------|:-------|:---------|:---------|
| 2-6                    | 10K                   | 50K    | 100K   | 500K     | 1000K    |
| CPU/Serial             | 6791                  | 168234 | 673806 | 16852984 | 67471402 |
| Original naive version | 65.3                  | 863    | 2884   | 63599    | 250574   |
| Original tiled version | 61.3                  | 714    | 2242   | 43843    | 168189   |
| Improved naive version | 27.9                  | 400    | 1366   | 31306    | 124353   |
| Improved tiled version | 21.0                  | 233    | 771    | 16797    | 66338    |

Execution time (/ms) of CPU and GPU versions of the AIDW algorithm on single precision

We have also calculated the speedups of our improved GPU-accelerated AIDW algorithm against the corresponding serial algorithm (i.e., the CPU version listed in Table ); see Figure . The results indicate that: (1) the highest speedups achieved by the naive version and the tiled version can be up to 543 and 1017, respectively; and (2) the tiled version is always faster than the naive version.

### Comparison of the Improved Naive Version and Tiled Version

As observed from the experimental tests, the tiled version of the improved algorithm is about 1.33 $\sim$ 1.87 times faster than the naive version. This behavior is due to the reason that: the stage of interpolating in the tiled version is much more computationally efficient than that in the naive version; see the execution time of the interpolating stage in Table .

As described in Section 3, the improved algorithm includes both the naive version and tiled version, which can be divided into two major stages: i.e., the stage of $k$NN search and the stage of weighted interpolating. The first stage in the above two versions are the same, while the second stage differs.

In the stage of interpolating of the tiled version, the benefit of the use of shared memory is exploited, while in the naive version it is not. For this reason, the interpolating stage in the tiled version executes about 1.79 $\sim$ 1.89 times faster than that in the naive version. Thus, the entire tiled version is more efficient than the naive version.

<table>
<caption>Execution time (/ms) of the stage of <span class="math inline"><em>k</em></span>NN search and the stage of weighted interpolating in the improved GPU-accelerated AIDW algorithm</caption>
<thead>
<tr>
<th style="text-align: left;">[0cm][0cm]<span>Stage</span></th>
<th colspan="5" style="text-align: center;">Data Size (1K = 1024)</th>
</tr>
</thead>
<tbody>
<tr>
<td style="text-align: left;"><span>2-6</span></td>
<td style="text-align: left;">10K</td>
<td style="text-align: left;">50K</td>
<td style="text-align: left;">100K</td>
<td style="text-align: left;">500K</td>
<td style="text-align: left;">1000K</td>
</tr>
<tr>
<td style="text-align: left;"><p><span class="math inline"><em>k</em></span>NN Search</p>
<p>(Both versions)</p></td>
<td style="text-align: left;">12.3</td>
<td style="text-align: left;">36</td>
<td style="text-align: left;">81</td>
<td style="text-align: left;">440</td>
<td style="text-align: left;">917</td>
</tr>
<tr>
<td style="text-align: left;"><p>Weighted Interpolating</p>
<p>(Improved naive version)</p></td>
<td style="text-align: left;">15.6</td>
<td style="text-align: left;">364</td>
<td style="text-align: left;">1286</td>
<td style="text-align: left;">30866</td>
<td style="text-align: left;">123437</td>
</tr>
<tr>
<td style="text-align: left;"><p>Weighted Interpolating</p>
<p>(Improved tiled version)</p></td>
<td style="text-align: left;">8.7</td>
<td style="text-align: left;">197</td>
<td style="text-align: left;">691</td>
<td style="text-align: left;">16357</td>
<td style="text-align: left;">65421</td>
</tr>
</tbody>
</table>

### Workload between the Stages of *k*NN Search and Weighted Interpolating

There are two major stages in the improved GPU-accelerated AIDW algorithm. To understand the efficiency bottleneck for further optimizations in the future, we in particular record the execution time for the stages of $k$NN search and weighted interpolating separately; see Table . In addition, we have also evaluated the workload percentage between the above two stages in both the naive version and tiled version; see Figure .

We have found that: the computational cost spent in the stage of $k$NN search is much less than that in the stage of the weighted interpolating. Moreover, with the increase of the size of testing data, the weight of the running time cost in the stage of $k$NN significantly decreases; and it even reduces to about one percentage. This observation indicates that most overhead in both the naive version and the tiled version is spent in the stage of weighted interpolating rather than the $k$NN search. Therefore, further optimizations may need to be employed to improve the efficiency of the weighted interpolating.

## Comparison with the Original GPU-accelerated AIDW Algorithm

In Section 5.1, we have evaluated the efficiency of the improved algorithm by comparing it with the serial AIDW algorithm, and found that our improved algorithm can achieve quite satisfied speedups. In this section, we will compare our improved GPU-accelerated algorithm presented in this work with the original GPU-accelerated algorithm introduced in .

The speedups of the improved algorithm over the original algorithm are illustrated in Figure . The results show that the improved naive version and tiled version are at least 2.02 and 2.54 times faster than the original naive version and tiled version, respectively. This also indicates that significant performance gains have been achieved by improving the original algorithm using fast $k$NN search.

The major difference between the original algorithm and the improved algorithm is the use of different $k$NN search approaches. We attempt to explain the reason why significant performance gains have been achieved by analyzing the impact of different $k$NN search algorithm on the computational efficiency.

First, we obtain the computational time of the $k$NN search in the original algorithm by subtracting the time spent in the stage of weighted interpolating from the total execution time; see Table . Note that, the execution time cost in the stage of weighted interpolating is directly derived from the improved algorithm. This is because that: (1) the weighted interpolating in both the original algorithm and the improved algorithm is the same; and (2) the running time of the weighted interpolating can be separately measured in the improved algorithm, while in contrast it is unable to accurately evaluate the execution time specifically for the weighted interpolating in the original algorithm.

| \[0cm\]\[0cm\]Version  | Data Size (1K = 1024) |     |      |       |        |
|:-----------------------|:----------------------|:----|:-----|:------|:-------|
| 2-6                    | 10K                   | 50K | 100K | 500K  | 1000K  |
| Original naive version | 49.7                  | 499 | 1598 | 32733 | 127137 |
| Original tiled version | 52.6                  | 517 | 1551 | 27486 | 102768 |
| Two improved versions  | 12.3                  | 36  | 81   | 440   | 917    |

Execution time (/ms) of the stage of $k$NN search in the original and the improved GPU-accelerated AIDW algorithm

Second, we calculate the percentages of the running time of the $k$NN search in the improved algorithm over that in the original algorithm; see Figure . We have found that: in both the naive version and the tiled version, the execution time of the $k$NN search in the improved algorithm is much less than that in the original algorithm, for example, less than one percentage for about one million points. This suggests that: the use of fast $k$NN search approach can significantly improve the efficiency of the entire GPU-accelerated AIDW interpolation algorithm.

# Conclusion

In this work, we have presented an efficient AIDW interpolation algorithm on the GPU by utilizing a fast $k$NN search method. The presented algorithm is composed of two major stages, i.e., the $k$NN search and weighted interpolating, and is developed by improving a previous GPU-accelerated AIDW algorithm with the use of fast $k$NN search. The $k$NN search is carried out based upon an even grid, and is capable of finding exact nearest neighbors very fast for each interpolated point. We have performed five groups of experimental tests to evaluate the performance of the improved GPU-accelerated AIDW algorithm. We have found that: (1) the improved algorithm can achieve a speedup of up to 1017 over the corresponding serial algorithm for one million points; (2) the improved algorithm is at least two times faster than our previously developed GPU-accelerated AIDW algorithm; and (3) the utilization of fast $k$NN search can significantly improve the computational efficiency of the entire GPU-accelerated AIDW algorithm. To benefit the community, all source code and testing data related to the presented AIDW algorithm is publicly available.

# Acknowledgments

This research was supported by the Natural Science Foundation of China (Grant No. 40602037 and 40872183), China Postdoctoral Science Foundation (2015M571081), and the Fundamental Research Funds for the Central Universities (2652015065). The authors would like to thank the editor and the reviewers for their contributions on the paper.

# References


abstract: We investigate the association between musical chords and lyrics by analyzing a large dataset of user-contributed guitar tablatures. Motivated by the idea that the emotional content of chords is reflected in the words used in corresponding lyrics, we analyze associations between lyrics and chord categories. We also examine the usage patterns of chords and lyrics in different musical genres, historical eras, and geographical regions. Our overall results confirms a previously known association between Major chords and positive valence. We also report a wide variation in this association across regions, genres, and eras. Our results suggest possible existence of different emotional associations for other types of chords.
address: $^{1}$ Department of Informatics, Indiana University, Bloomington, IN 47408, United States
  $^{2}$ Santa Fe Institute, Santa Fe, NM 87501, United States (current affiliation)
  $^{3}$ Amazon.com
  $^{4}$ Epic.com
  $^{+}$ these authors contributed equally to this work
author: Artemy Kolchinsky$^{1,2,+}$, Nakul Dhande$^{1,3+}$, Kengjeun Park$^{1,4}$, Yong-Yeol Ahn$^{1}$
bibliography: main.bib
title: The Minor Fall, the Major Lift: Inferring Emotional Valence of Musical Chords through Lyrics; The Minor Fall, the Major Lift: Inferring Emotional Valence of Musical Chords through Lyrics

The study of music perception has been dominated by methods that *directly* measure emotional responses, such as self-reports, physiological and cognitive measurements, and developmental observations . Such methods may produce high-quality data, but the data collection process involved is both labor- and resource-intensive. As a result, creating large datasets and discovering statistical regularities has been a challenge.

Meanwhile, the growth of music databases  as well as the advancement of the field of Music Information Retrieval (MIR)  opened new avenues for data-driven studies of music. For instance, sentiment analysis  has been applied to uncover a long-term trend of declining valence in popular song lyrics . It has been shown that the lexical features from lyrics , metadata , social tags , and audio-based features can be used to predict the mood of a song. There has been also an attempt to examine the associations between lyrics and individual chords using a machine translation approach, which confirmed the notion that Major and Minor chords are associated with happy and sad words respectively .

Here, we propose a novel method to study the associations between chord types and emotional valence. In particular, we use sentiment analysis to analyze *chord categories* (e.g. Major and Minor) and their associations with sentiment and words across genres, regions, and eras.

To do so, we adopt a sentiment analysis method that uses the crowd-sourced *LabMT 1.0* valence lexicon . Valence is one of the two basic emotional axes , with higher valence corresponding to more attractive/positive emotion. The lexicon contains valence scores ranging from $0.0$ (saddest) to $9.0$ (happiest) for $10,222$ common English words obtained by surveying Amazon's Mechanical Turk workers. The overall valence of a piece of text, such as a sentence or document, is measured by averaging the valence score of individual words within the text. This method has been successfully used to obtain insight into a wide variety of corpora .

Here, we apply this sentiment analysis method to a dataset of guitar tablatures — which contain both lyrics and chords — extracted from ultimate-guitar.com. We collect all words that appear with a specific chord and create a large "bag of words" — a frequency list of words — for each chord (see Fig. ). We perform our analysis by aggregating chords based on their 'chord category' (e.g., Major chords, Minor chords, Dominant 7th chords, etc.). In addition, we also acquire metadata from the Gracenote API regarding the genre of albums, as well as era and geographical region of musical artists. We then perform our analysis of associations between lyrics sentiment and chords within the categories of genre, era, and region. Details of our methodology are described in the next section.

# Materials and Methods

Guitar tabs were obtained from ultimate-guitar.com , a large online user-generated database of tabs, while information about album genre, artist era, and artist region was obtained from Gracenote API , an online musical metadata service.

## Chords-lyrics association

`ultimate-guitar.com` is one of the largest user-contributed tab archives, hosting more than 800,000 songs. We examined 123,837 songs that passed the following criteria: (1) we only kept guitar tabs and ignored those for other instruments such as the ukulele; (2) we ignored non-English songs (those having less than half of their words in an English word list  or identified as non-English by the `langdetect` library ); (3) when multiple tabs were available for a song, we kept only the one with the highest user-assigned rating. We then cleaned the raw HTML sources and extracted chords and lyrics transcriptions. As an example, Fig.  shows how the tablature of Leonard Cohen's "Hallelujah"  is processed to produce a chord-lyrics table.

Sometimes, chord symbols appeared in the middle of words; in such cases, we associated the entire words with the chord that appears in its middle, rather than the previous chord. In addition, chords that could not be successfully parsed or that had no associated lyrics were dropped.

## Metadata collection using Gracenote API

We used the Gracenote API (<http://gracenote.com>) to obtain metadata about artists and song albums. We queried the title and the artist name of the 124,101 songs that were initially obtained from `ultimate-guitar.com`, successfully retrieving Gracenote records for 89,821 songs. Songs that did not match a Gracenote record were dropped. For each song, we extracted the following metadata fields:

-   The geographic **region** from which the artist originated (e.g. *North America*). This was extracted from the highest-level geographic labels provided by Gracenote.

-   The musical **genre** (e.g. *50's Rock*). This was extracted from the second-level genre labels assigned to each album by Gracenote.

-   The historical **era** at the level of decades (e.g. *1970's*). This was extracted from the first-level era labels assigned to each artist by Gracenote. Approximately 6,000 songs were not assigned to an era, in which case they were assigned to the decade of the album release year as specified by Gracenote.

In our analysis, we reported individual statistics only for the most popular regions (*Asia*, *Australia/Oceania*, *North America*, *Scandinavia*, *Western Europe*), genres (top 20 most popular genres), and eras (1950's through 2010's).

## Determining chord categories

We normalized chord names and classified them into chord categories according to chord notation rules from an online resource . All valid chord names begin with one or two characters indicating the root note (e.g. `G` or `Bb`) which are followed by characters which indicate the *chord category*. We considered the following chord categories :

-   **Major**: A Major chord is a triad with a root, a major third and a perfect fifth. Major chords are indicated using either only the root note, or the root note followed by `M` or `maj`. For instance, `F`, `FM`, `G`, `Gmaj` were considered Major chords.

-   **Minor**: A Minor chord is also a triad, containing a root, minor third and a perfect fifth. The notation for Minor chords is to have the root note followed by `m` or `min`. For example, `Emin`, `F#m` and `Bbm` were considered Minor chords.

-   **7th**: A seventh chord has seventh interval in addition to a Major or Minor triad. A **Major 7th** consists of a Major triad and an additional Major seventh, and is indicated by the root note followed by `M7` or `maj7` (e.g. `GM7`). A **Minor 7th** consists of a Minor triad with an additional Minor seventh makes, and is indicated by the root note followed by `m7` or `min7` (e.g. `Fm7`). A **Dominant 7th** is a diatonic seventh chord that consists of a Major triad with additional Minor seventh, and is indicated by the root note followed by the numeral `7` or `dom7` (e.g. `D7`, `Gdom7`).

-   **Special chords with '\*'**: In tab notation, the asterisk '\*' is used to indicate special instructions and can have many different meanings. For instance, `G*` may indicate that the `G` should be played with a palm mute, with a single strum, or some other special instruction usually indicated in free text in the tablature. Because in most cases the underlying chord is still played, in this study we map chords with asterisks to their respective non-asterisk versions. For instance, we consider `G*` to be the same as `G` and `C7*` to be the same as `C7`.

-   **Other chords**: There were several other categories of chords that we do not analyze individually in this study. One of these is **Power** chords, which are dyads consisting of a root and a perfect fifth. Because Power chords are highly genre specific, and because they sometimes function musically as Minor or Major chords, we eliminated them from our study. For reasons of simplicity and statistical significance, we also eliminated several other categories of chords, such as **Augmented** and **Diminished** chords, which appeared infrequently in our dataset.

In total, we analyzed 924,418 chords (see the next subsection). Figure  shows the prevalence of different chord categories among these chords.

## Sentiment analysis

Sentiment analysis was used to measure the valence (happiness vs. unhappiness) of chord lyrics. We employed a simple methodology based on a crowd-sourced valence lexicon (`LabMT 1.0`) . This method was chosen because (1) it is simple and scalable (2) it is *transparent*, allowing us to calculate the contribution from each word to the final valence score, and (3) it has been shown to be useful in many studies . The `LabMT 1.0` lexicon contains valence scores ranging from 0.0 (saddest) to 9.0 (happiest) for 10,222 common English words as obtained by surveying Amazon's Mechanical Turk workers. The valence assigned to some sequence of words (e.g. words in the lyrics corresponding to Major chords) was computed by mapping each word to its corresponding valence score and then computing the mean. Words not found in the LabMT lexicon were ignored; in addition, following recommended practices for increasing sentiment signal , we ignored emotionally-neutral words having a valence strictly between 3.0 and 7.0. Chords that were not associated with any sentiment-mapped words were ignored. The final dataset contained 924,418 chords from 86,627 songs.

## Word shift graphs

In order to show *how* a set of lyrics (e.g. lyrics corresponding to songs in the *Punk* genre) differs from the overall lyrics dataset, we use the *word shift graphs* . We designate the whole dataset as the *reference* (baseline) corpus and call the set of lyrics that we want to compare *comparison corpus*. The difference in their overall valence can now be broken down into the contribution from each individual word. Increased valence can result from either having a higher *prevalence* (frequency of occurrence) of high-valence words or a lower prevalence of low-valence words. Conversely, lower valence can result from having a higher prevalence of low-valence words or a lower prevalence of high-valence words. The percentage contribution of an individual word $i$ to the valence difference between a comparison and reference corpus can be expressed as: $$\begin{aligned}
%
100\cdot\frac{\overbrace{\left(h_i-h^{\textrm{(ref)}}\right)}^{+/-}\overbrace{\left(p_{i}-p_{i}^{\textrm{(ref)}}\right)}^{\uparrow/\downarrow}}{\left|h^{\textrm{(comp)}}-h^{\textrm{(ref)}}\right|}
%
\end{aligned}$$ where $h_i$ is the valence score of word $i$ in the lexicon, $h^{\textrm{(ref)}}$ and $h^{\textrm{(comp)}}$ are the mean valences of the words in the reference corpus and comparison corpus respectively, $p_{i}$ is the normalized frequency of word $i$ in the comparison corpus, and $p_{i}^{\textrm{(ref)}}$ is the normalized frequency of word $i$ in the reference corpus (normalized frequencies are computed as $p_i = \frac {n_{i}}{\sum_{i'}{n_{i'}}}$, where $n_i$ is the number of occurrences of a word $i$). The first term (indicated by '+/-') measures the difference in word valence between word $i$ and the mean valence of the reference corpus, while the second term (indicated by $\uparrow/\downarrow$) looks at the difference in word prevalence between the comparison and reference corpus. In plotting the word shift graphs, for each word we use +/- signs and blue/green bar colors to indicate the (positive or negative) sign of the valence term and $\uparrow/\downarrow$ arrows to indicate the sign of the prevalence term.

## Model comparison

In the Results section, we evaluate what explanatory factors (chord category, genre, era, and region) best account for differences in valence scores. Using the `statsmodels` toolbox , we estimated linear regression models where the mean valence of each chord served as the response variable and the most popular chord categories, genres, eras, and regions served as the categorical predictor variables. The variance of the residuals was used to compute the proportion of variance explained when using each factor in turn.

We also compared models that used combinations of factors. As before, we fit linear models that predicted valence. Now, however, explanatory factors were added in a greedy fashion, with each additional factor to minimize the Aikake information criterion (AIC) of the overall model.

# Results

## Valence of chord categories

We measure the mean valence of lyrics associated with different chord categories (Figure A). We find that major chords have higher valence than Minor chords, concurring with numerous studies which argue that human subjects perceive Major chords as more emotionally positive than Minor chords . However, our results suggest that Major chords are not the happiest: all three categories of 7th chords considered here (Minor 7th, Major 7th, and Dominant 7th) exhibit higher valence than Major chords. This effect holds with high significance ($p \ll 10^{-3}$ for all, one-tailed Mann-Whitney tests).

In Fig. B-F, we use the word shift graphs  to identify words that contribute most to the difference between the valence of each chord category and baseline (mean valence of all lyrics in the dataset). For instance, "lost" is a lower-valence word (blue color and '-' sign) that is underexpressed in Major chords ('$\downarrow$' sign), increasing the mean valence of Major chords above baseline. Many negative words, such as "pain", "fight", "die", and "lost" are overexpressed in Minor chord lyrics and underrepresented in Major chord lyrics.

Although the three types of 7th chords have similar valence scores (Fig. A), word shift graphs reveals that they may have different "meanings" in terms of associated words. Overexpressed high-valence words for Dominant 7th chords include terms of endearment or affection, such as "baby", "sweet", and "good" while for Minor 7th chords they are "life" and "god". Lyrics associated with Major 7th chords, on the other hand, have a stronger under-representation of negative words (e.g. "die" and "hell").

## Genres

We analyze the emotional content and meaning of lyrics from albums in different musical genres. Fig. A shows the mean valence of different genres, demonstrating that an intuitive ranking emerges when genres are sorted by valence: *Religious* and *60's Rock* lyrics reside at the positive end of the spectrum while *Metal* and *Punk* lyrics appear at the most negative.

As mentioned in the previous section, Minor chords have a lower mean valence than Major chords. We computed the numerical *differences* in valence between Major and Minor chords for albums in different genres (Fig. B). All considered genres, with the exception of *Contemporary R&B/Soul*, had a mean valence of Major chords higher than that of Minor chords. Some of the genres with the largest Major vs. Minor valence differences include *Classic R&B/Soul*, *Classic Country*, *Religious*, and *60's Rock*.

We show word shift graphs for two musical genres: *Religious* (Fig. C) and *Punk* (Fig. D). The highest contributions to the *Religious* genre come from the overexpression of high-valence words having to do with worship ("love", "praise", "glory", "sing"). Conversely, the highest contributions to the *Punk* genre come from the overexpression of low-valence words (e.g. "dead", "sick", "hell"). Some exceptions exist: for example, *Religious* lyrics underexpress high-valence words such as "baby" and "like", while *Punk* lyrics underexpress the low-valence word "cry".

## Era

In this section, we explore sentiment trends for artists across different historical eras. Fig. A shows the mean valence of lyrics in different eras. We find that valence has steadily decreased since the 1950's, confirming results of previous sentiment analysis of lyrics , which attributed the decline to the recent emergence of 'dark' genres such as metal and punk. However, our results demonstrate that this trend has recently undergone a reversal: lyrics have higher valence in the 2010's era than in the 2000's era.

As in the last section, we analyze differences between Major and Minor chords for lyrics belonging to different eras (Fig. B). Although Major chords have a generally higher valence than Minor chords, surprisingly this distinction does not hold in the 1980's era, in which Minor and Major chord valences are similar. The genres in the 1980's that had Minor chords with higher mean valence than Major chords — in other words, which had an 'inverted' Major/Minor valence pattern — include *Alternative Folk*, *Indie Rock*, and *Punk* (data not shown).

Finally, we report changes in chord usage patterns across time. Fig. C shows the proportion of chords belonging to each chord category in different eras (note the logarithmic scale). Since the 1950's, Major chord usage has been stable while Minor chords usage has been steadily growing. Dominant 7th chords have become less prevalent, while Major 7th and Minor 7th chords had an increase in usage during the 1970's.

The finding that Minor chords have become more prevalent while Dominant 7th chords have become rarer agrees with a recent data-driven study of the evolution of popular music genres . The authors attribute the latter effect to the decline in the popularity of blues and jazz, which frequently use Dominant 7th chords. However, we find that this effect holds widely, with Dominant 7th chords diminishing in prevalence even when we exclude genres associated with Blues and Jazz (data not shown). More qualitatively, musicologists have argued that many popular music styles in the 1970's exhibited a decline in the use of Dominant 7th chords and a growth in the use of Major 7th and Minor 7th chords  — clearly seen in the corresponding increases in Fig. C.

## Region

In this section, we evaluate the emotional content of lyrics from artists in different geographical regions. Fig. A shows that artists from Asia have the highest-valence lyrics, followed by artists from *Australia/Oceania*, *Western Europe*, *North America*, and finally *Scandinavia*, the lowest valence geographical region. The latter region's low valence is likely due to the over-representation of 'dark' genres (such as metal) among Scandinavian artists .

As in previous sections, we compare differences in valence of Major and Minor chords for different regions (Fig. B). All regions except *Asia* have a higher mean valence for Major chords than Minor chords, while for the *Asian* region there is no significant difference.

There are several important caveats to our geographical analysis. In particular, our dataset consisted of only English-language songs, and is thus unlikely to be representative of overall musical trends in non-English speaking countries. This bias, along with others, is discussed in more depth in the Discussion section.

## Model comparison

We have shown that mean valence varies as a function of chord category, genre, era, and region (which are here called *explanatory factors*). We evaluate what explanatory factors best account for differences in valence scores. Fig. A shows the proportion of variance explained when using each factor in turn as a predictor of valence. This shows that genre explains most variation in valence, followed by era, chord category, and finally region.

It is possible that variation in valence associated with some explanatory factors is in fact 'mediated' by other factors. For example, we found that mean valence declined from the 1950's era through the 2000's, confirming previous work  that explained this decline by the growing popularity of 'dark' genres like *Metal* and *Punk* over time; this is an example in which valence variation over historical eras is argued to actually be attributable to variation in the popularities of different genres. As another example, it is possible that Minor chords are lower valence than Major chords because they are overexpressed in dark genres, rather than due to their inherent emotional content.

We investigate this effect using statistical model selection. For instance, if the valence variation over chord categories can be entirely attributed to genre (i.e. darker genres have more Minor chords), then model selection should prefer a model that contains only the genre explanatory factor to the one that contains both the genre and chord category explanatory factors.

We fit increasingly large models while computing their Aikake information criterion (AIC) scores, a model selection score (lower is better). As Fig. B shows, the model that includes all four explanatory factors has the lowest AIC, suggesting that chord category, genre, era, and region are all important factors for explaining valence variation.

# Discussion

In this paper, we propose a novel data-driven method to uncover emotional valence associated with different chords as well as different geographical regions, historical eras, and musical genres. We then apply it to a novel dataset of guitar tablatures extracted from ultimate-guitar.com along with musical metadata provided by the Gracenote API. We use word shift graphs to characterize the meaning of chord categories as well as categories of lyrics.

We find that Major chords are associated with higher valence lyrics than Minor chords, consistent with the previous music perception studies that showed that Major chords evoke more positive emotional responses than Minor chords . For an intuition regarding the magnitude of the difference, the mean valence of Minor chord lyrics is approximately $6.16$ (e.g. the valence of the word "people" in our sentiment dictionary), while the mean valence of Major chord lyrics is approximately $6.28$ (e.g. the valence of the word "community" in our sentiment dictionary). Interestingly, we also uncover that three types of 7th chords — Dominant 7ths, Major 7ths, and Minor 7ths — have even higher valence than Major chords. This effect has not been deeply researched, except for one music perception study which reported that, in contrast to our findings, 7th chords evoke emotional responses intermediate in valence between Major and Minor chords .

Significant variation exists in the lyrics associated with different geographical regions, musical genres, and historical eras. For example, musical genres demonstrate an intuitive ranking when ordered by mean valence, ranging from low-valence *Punk* and *Metal* genres to high-valence *Religious* and *60's Rock* genres. We also found that sentiment declined over time from the 1950's until the 2000's. Both of these findings are consistent with the results of a previous study conducted using a different dataset and lexicon . At the same time, we report a new finding that the trend in declining valence has reversed itself, with lyrics from the 2010's era having higher valence than those from the 2000's era. Finally, we perform a analysis of the variation of valence among geographical regions. We find that *Asia* has the highest valence while *Scandinavia* has the lowest (likely due to prevalence of 'dark' genres in that region).

We perform a novel data-driven analysis of the Major vs. Minor distinction by measuring the *difference* between Major and Minor valence for different regions, genres, and eras. All examined genres except *Contemporary R&B/Soul* exhibited higher Major chord valence than Minor chord. Interestingly, the largest differences of Major above Minor may indicate genres (*Classic R&B/Soul*, *Classic Country*, *Religious*, *60's Rock*) that are more musically 'traditional'. In terms of historical periods, we find that, unlike other eras, the 1980's era did not have a significant Major-Minor valence difference. This phenomenon calls for further investigation; one possibility is that it may be related to an important period of musical innovation in 1980's, which was recently reported in a data-driven study of musical evolution . Finally, analysis of geographic variation indicates that songs from the *Asian* region—unlike those from other regions—do not show a significant difference in the valence of Major vs. Minor chords. In fact, it is known in the musicological literature that the association of positive emotions with Major chords and negative emotions with Minor chords is culture-dependent, and that some Asian cultures do not display this association . Our results may provide new supporting evidence of cultural variation in the emotional connotations of the Major/Minor distinction.

Finally, we evaluate how much of the variation in valence in our dataset is attributable to chord category, genre, era, and region (we call these four types of attributes 'explanatory factors'). We find that genre is the most explanatory, followed by era, chord category, and region. We use statistical model selection to evaluate whether certain explanatory factors 'mediate' the influence of others (an example of mediation would be if variation in valence of different eras is actually due to variation in the prevalence of different genres during those eras). We find that all four explanatory factors are important for explaining variation in valence; no explanatory factors totally mediate the effect of others.

Our approach has several limitations. First, the accuracy of tablature chord annotations may be limited because users are not generally professional musicians; for instance, more complex chords (e.g. `dim` or `11th`) may be mis-transcribed as simpler chords. To deal with this, we analyze relatively basic chords—Major, Minor, and 7ths—and, when there are multiple versions of a song, use tabs with the highest user-assigned rating. Our manual inspection of a small sample of parsed tabs indicated acceptable quality, although a more systematic evaluation of the error rate can be performed using more extensive manual inspection of tabs by professional musicians.

There are also significant biases in our dataset. We only consider tablatures for songs entered by users of `ultimate-guitar.com`, which is likely to be heavily biased towards North American and European users and songs. In addition, this dataset is restricted to songs playable by guitar, which selects for guitar-oriented musical genres and may be biased toward emotional meanings tied to guitar-specific acoustic properties, such as the instrument's timbre. Furthermore, our dataset includes only English-language songs and is not likely to be a representative sample of popular music from non-English speaking regions. Thus, for example, the high valence of songs from *Asia* should not be taken as conclusive evidence that Asian popular music is overall happier than popular music in English-speaking countries. For this reason, the absence of a Major vs. Minor chord distinction in *Asia* is speculative and requires significant further investigation.

Despite these limitations, we believe that our results reveal meaningful patterns of association between music and emotion at least in guitar-based English-language popular music, and show the potential of our novel data-driven methods. At the same time, applying these methods to other datasets — in particular those representative of other geographical regions, historical eras, instruments, and musical styles — is of great interest for future work. Another promising direction for future work is to move the analysis of emotional content beyond single chords, since emotional meaning is likely to be more closely associated with melodies rather than individual chords. For this reason, we hope to extend our methodology to study chord progressions.