Facet versus scale prediction of Big 5 personality NEO facets and scales: The NEO model of personality has 5 scales (i.e., extraversion, neuroticism, agreeableness, openness, and conscientiousness). For each scale there are 6 facets (e.g., neuroticism has anxiety, hostility, depression, self-consciousness, impulsiveness, and vulnerability to stress). See here for a list of the 30 facets and how they relate to the 5 scales.

Incremental prediction: I am interested in the degree to which the 30 facets predict important real-world outcomes over and above the big 5 scales. Examples of important outcomes might include financial success, life satisfaction, health outcomes, longevity, job performance, career success, marital satisfaction, criminality, and so on. While each of these outcomes would be interesting in itself, I'm particularly interested in any general claims that can be put forward regarding the incremental predictive validity that is obtained from using facets rather than scales.

Statistical baseline: I take a few points as self-evident. There is a difference between in sample prediction and true population prediction. The true population regression equation can not be worse for the facets than the scales. The facet level prediction could mirror the scale prediction merely through an appropriate set of regression coefficients (i.e., some form where facets are in some sense equal within a scale, adding up to the scale effect).

I also take it as self-evident that the population prediction would be at least a little bit better with facets. It seems implausible that the pattern of regression coefficients required to replicate the scale prediction would necessarily be exactly match the best population prediction model. However, there is still the question of what the size of the increase is, and how big the increase in prediction is relative to the prediction achieved by the scales.

Methodological challenges: There are also several issues around estimating the size of the incremental prediction of facets relative to scales. Adding an additional predictor increases sample variance explained even when the predictor has know incremental prediction in the population. Thus, procedures that compare sample r-squared of scales versus facets prediction models would be biased. Models that use a data driven variable selection procedure like step-wise regression to select an optimal subset of predictors would also be biased. The size of this bias would be greatest in smaller samples. 

Thus, I'm particularly interested in studies that have accounted for these methodological challenges. For example, a good study might have (a) a large sample of 1000+ participants in order to generate robust results, and (b) a statistically valid procedure for estimating population r-squared for facet and scale predictor models (one possibility would be to estimate the model in half the sample and evaluate in the other half, although many other cross-validation procedures are possible).

Thus, my questions:


What is the incremental prediction of the 30 facets over and above the big 5 scales on important psychological outcomes?
In particular, how and to what extent have studies dealt with the methodological issues related to performing this comparison?
In totality, is the incremental prediction at the facet level sufficient to justify the added complexity to thinking and theorising?
Several studies have been conducted comparing facets versus factor prediction (e.g., Paunonen and Ashton, 2001; Mershon and Gorsuch, 1988; Ashton et al, 1995; Dudley et al, 2006; Ekehammar &amp; Akrami, 2007).

Many of these studies have found substantial incremental prediction of facets over factors. However, there are several potential methodical issues that may potentially arise: (a) overcoming biased estimates of R-square particularly when comparing analyses with facets versus factors where more predictors increases the bias; (b) overcoming issues of multiple comparisons when there are many facets that can be compared to factors; (c) some studies only have small sample sizes. 

By statistical definition, the population r-square for a factor model is less than or equal to the population r-square for a facet model where factors are defined as a linear function of the facets. Nonetheless, I have not yet read any systematic studies that have provided an unbiased estimate of population r-square in factors versus facets. Having read all the above articles, my hypothesis would be that I would expect an increase in r-square of around 5 to 20 percent, but that's just a rough sense. 

References


Ashton, M. C., Jackson, D. N., Paunonen, S. V., Helmes, E., &amp; Rothstein, M. G. (1995). The criterion validity of broad factor scales versus specific facet scales. Journal of Research in Personality, 29(4), 432-442.
Dudley, N. M., Orvis, K. A., Lebiecki, J. E., &amp; Cortina, J. M. (2006). A meta-analytic investigation of conscientiousness in the prediction of job performance: examining the intercorrelations and the incremental validity of narrow traits. Journal of Applied Psychology, 91(1), 40.
Ekehammar, B., &amp; Akrami, N. (2007). Personality and prejudice: From Big Five personality factors to facets. Journal of Personality, 75(5), 899-926.
Mershon, B., &amp; Gorsuch, R. L. (1988). Number of factors in the personality sphere: Does increase in factors increase predictability of real-life criteria?. Journal of Personality and Social Psychology, 55(4), 675.
Paunonen, S. V., &amp; Ashton, M. C. (2001). Big Five factors and facets and the prediction of behavior. Journal of personality and social psychology, 81(3), 524-539.PDF