PMC4288287_01
PMC4288287_02
The score of "Traumerei," used by all the pianists. Section IDs (boxed letters) are added for this Figure. This image was reproduced by the first author.
fpsyg-05-01564-g0001.jpg
PMC4288287_02_fpsyg-05-01564-g0001.jpg
F1
We selected six Western-classical pieces as the materials, including Traumerei (Figure 1). Traumerei varies in accordance with the hierarchical structure of the piece: Pianists frequently lengthen accented tones within melodic gestures, and this lengthening is more exaggerated at the end of each section in accordance with the composer's ritardando marking (i.e., A, rep-A, B, A'; see Figure 1). The degree of lengthening is the greatest at the end of the piece (i.e., the end of the "highest" level in the structural hierarchy of the piece). For example, their third component showed the typical expression at the transitional point from sections B to A' or the duration of the first fermata in section A' (see Figure 1). Results showed three components (PCs; Supplementary Figure S1) in the smoothed beat duration that were responsible for these durational variations. For example, in Figure 5, PC 1 score of the audience-present performance by Pianist 11 was the highest, meaning that he performed this piece in the slowest tempo (i.e., 48.88 bpm based on the mean beat duration) among the pianists. The most prominent role of PC 3 (Figure 4C) was the extreme reduction of the value at fermata before the final ritardando (see Figure 1, measure 22, q = 116), resulting in a small range of the variation in section A'.
Examples of the raw data (A) and the cumulative value (B) for the beat duration of the audience-present performance by Pianist 11. The letters within each graph indicate section IDs. The x-axis indicates the number of quarter note.
fpsyg-05-01564-g0002.jpg
PMC4288287_02_fpsyg-05-01564-g0002.jpg
F2
Figure 2 shows an example of the raw data and the cumulative values by Pianist 11, who showed the clearest difference between the audience-present and the audience-absent conditions. We applied local polynomial regression with degree two (see Fan and Gijbels,) for the cumulative values of each performance (Figure 2B) and calculated the first derivative as the smoothed beat duration (Figure 3 as an example). We conducted a functional principal components analysis to the smoothed dynamic range curve for each recording, extracting three principal components (Supplementary Figure S2).
Examples of the smoothed beat duration curves for the audience-present performance by Pianist 11. The letters within the graph indicate section IDs. The x-axis indicates the number of quarter note.
fpsyg-05-01564-g0003.jpg
PMC4288287_02_fpsyg-05-01564-g0003.jpg
F3
We applied local polynomial regression with degree two (see Fan and Gijbels,) for the cumulative values of each performance (Figure 2B) and calculated the first derivative as the smoothed beat duration (Figure 3 as an example). As shown in Figure 3, the beat duration was slower in section A' (the end of the piece) than in the other sections, indicating that the pianist modulated the beat duration based on the overall structure of the piece.
An example of how each component functioned in the smoothed beat duration of the audience-present performance, based on Pianist 11's data. The role of each component can be extracted by adding the value of the target component to the original curve. The letters within each graph indicate section IDs. The x-axis indicates the number of quarter note.
fpsyg-05-01564-g0004.jpg
PMC4288287_02_fpsyg-05-01564-g0004.jpg
F4
Figure 4 shows an example of what each component contributed to the smoothed beat duration by adding each principal component to the original curve. Because the synthesis of the three components was fairly identical to the original curve (Figure 4D), we can conclude that the beat duration was explained to a large extent by these components. More specifically, PC 1 generally increased the values of the original curve (Figure 4A); the role of PC 1 can be regarded as the amplification of the overall beat duration. When PC 2 was added (Figure 4B), the values in sections A, rep-A, and B were relatively consistent to the original curve, but those in section A' were enhanced, especially at the very end. The most prominent role of PC 3 (Figure 4C) was the extreme reduction of the value at fermata before the final ritardando (see Figure 1, measure 22, q = 116), resulting in a small range of the variation in section A'.
Relationships of the component scores between PCs 1 (the overall mean) and 2 (the cross-sectional contrast between the final and the remaining sections) (A) and between PCs 1 and 3 (the control of overall variability) (B) for the smoothed beat duration. The arrows indicate the directions changed from the audience-absent ( ) to the audience-present ( ) conditions. The numerals in the plots are pianist IDs. (PC: principal component).
fpsyg-05-01564-g0005.jpg
PMC4288287_02_fpsyg-05-01564-g0005.jpg
F5
For example, in Figure 5, PC 1 score of the audience-present performance by Pianist 11 was the highest, meaning that he performed this piece in the slowest tempo (i.e., 48.88 bpm based on the mean beat duration) among the pianists. The component score in each condition for each pianist was scatter-plotted in Figure 5.
Examples of the original dynamic range (A) and the smoothed curve as the smoothed dynamic range (B) for the audience-present performance by Pianist 11. The letters within each graph indicate sections IDs. The x-axis indicates the number of quarter note.
fpsyg-05-01564-g0006.jpg
PMC4288287_02_fpsyg-05-01564-g0006.jpg
F6
For the dynamic range (Figure 6A), we also applied the same time-series analyses. An example of the smoothed curve (Figure 6B) indicates the degree of change in dynamic range as a function of beat; we shall call this curve as a "smoothed dynamic range" curve. In Figure 6B, the general patterns in sections A, rep-A, and A' were similar, in that the highest peak exists around two third of each section.
An example of how each component functioned in the smoothed dynamic range curve of the audience-present performance, based on Pianist 11's data. The role of each component can be extracted by adding the value of the target component to the original curve. The letters within each graph indicate section IDs. The x-axis indicates the number of quarter note.
fpsyg-05-01564-g0007.jpg
PMC4288287_02_fpsyg-05-01564-g0007.jpg
F7
Figure 7 shows an example of how each of the components functioned in the smoothed dynamic range curve by adding each component to the original curve. As can be seen in Figure 7D, the synthesis of all the components was fairly identical to the original curve, validating that the smoothed dynamic range curve was explained to a large extent by these three components. By adding PC 1 to the original smoothed dynamic range curve, the value increased throughout the piece (Figure 7A), meaning that PC 1 enhanced the degree of the smoothed dynamic range curve. PC 2 (Figure 7B) appeared to reflect the variation in the smoothed dynamic range curve. The tendency of PC 3 (Figure 7C) appeared to differ as a function of the section (i.e., A/rep-A, B, A').
Relationships of the component scores between PCs 1 (overall mean) and 2 (the control of overall variability) (A) and between PCs 1 and 3 (the cross-sectional contrast between the final and the remaining sections) (B) for the smoothed dynamic range curve. The arrows indicate the directions changed from the audience-absent ( ) to the audience-present ( ) conditions. The numerals in the plots are pianist IDs. (PC: principal component).
fpsyg-05-01564-g0008.jpg
PMC4288287_02_fpsyg-05-01564-g0008.jpg
F8
The component score in each condition for each pianist was scatter-plotted in Figure 8.
Theoretically possible models for the relationship between the performer's expressions and the listener's responses. The blue paths and correlations (bilateral arrows) indicate the common relationships regardless of the level of listeners' musical training, while the red paths and correlations indicate the relationships differing as a function of the level of musical training. The variables e1 and e2 are error variables.
fpsyg-05-01564-g0009.jpg
PMC4288287_02_fpsyg-05-01564-g0009.jpg
F9
Figure 9 shows three theoretically possible models. In model A (Figure 9A), both listeners' responses (i.e., quality, emotionally moving experience) are predicted by both principal components. Model B (Figure 9B), pianists' expressions determine listeners' judgment of the quality, which subsequently predicts their emotionally moving experience. Model C (Figure 9C) is a reversed model of model B. Thus, model A (Figure 9A) was the best of the three.
The final model of the relationship between the performer's expressions and the listener's responses. The variables e1 and e2 are error variables.
fpsyg-05-01564-g0010.jpg
PMC4288287_02_fpsyg-05-01564-g0010.jpg
F10
N = 13) = 5.94, p = 0.97; GFI = 0.92, AGFI = 0.82, RMSEA = 0.001. Note that the coefficients described in Figure 10 were standardized, so that the values can be different across the training levels even for the paths specifying "the common relationship regardless of training (i.e., blue lines in Figure 10)." According to Figure 10, the durational and the dynamic parameters correlated positively with each other, indicating that pianists' inclinations toward "averaged" expressions are common between the parameters. The most experienced listeners were moved more when the smoothed dynamic range between the final and the remaining sections was less averaged (Figure 10C), whereas the effect was reversed for the other listeners (Figures 10A,B).