Documentation

### This is machine translation

Translated by
Mouseover text to see original. Click the button below to return to the English version of the page.

## Linear Mixed-Effects Models

Linear mixed-effects models are extensions of linear regression models for data that are collected and summarized in groups. These models describe the relationship between a response variable and independent variables, with coefficients that can vary with respect to one or more grouping variables. A mixed-effects model consists of two parts, fixed effects and random effects. Fixed-effects terms are usually the conventional linear regression part, and the random effects are associated with individual experimental units drawn at random from a population. The random effects have prior distributions whereas fixed effects do not. Mixed-effects models can represent the covariance structure related to the grouping of data by associating the common random effects to observations that have the same level of a grouping variable. The standard form of a linear mixed-effects model is

$y=\underset{fixed}{\underbrace{X\beta }}+\underset{random}{\underbrace{Zb}}+\underset{error}{\underbrace{\epsilon }},$

where

• y is the n-by-1 response vector, and n is the number of observations.

• X is an n-by-p fixed-effects design matrix.

• β is a p-by-1 fixed-effects vector.

• Z is an n-by-q random-effects design matrix.

• b is a q-by-1 random-effects vector.

• ε is the n-by-1 observation error vector.

The assumptions for the linear mixed-effects model are:

• Random-effects vector, b, and the error vector, ε, have the following prior distributions:

$\begin{array}{l}b~N\left(0,{\sigma }^{2}D\left(\theta \right)\right),\\ \epsilon ~N\left(0,\sigma {}^{2}I\right),\end{array}$

where D is a symmetric and positive semidefinite matrix, parameterized by a variance component vector θ, I is an n-by-n identity matrix, and σ2 is the error variance.

• Random-effects vector, b, and the error vector, ε, are independent from each other.

Mixed-effects models are also called multilevel models or hierarchical models depending on the context. Mixed-effects models is a more general term than the latter two. Mixed-effects models might include factors that are not necessarily multilevel or hierarchical, for example crossed factors. That is why mixed-effects is the terminology preferred here. Sometimes mixed-effects models are expressed as multilevel regression models (first level and grouping level models) that are fit simultaneously. For example, a varying or random intercept model, with one continuous predictor variable x and one grouping variable with M levels, can be expressed as

$\begin{array}{l}{y}_{im}={\beta }_{0m}+{\beta }_{1}{x}_{im}+{\epsilon }_{im},\text{ }i=1,2,..,n,\text{ }m=1,2,...,M,\text{ }{\epsilon }_{im}~N\left(0,{\sigma }^{2}\right),\\ {\beta }_{0m}={\beta }_{00}+{b}_{0m},\text{ }{b}_{0m}~N\left(0,{\sigma }_{0}^{2}\right),\end{array}$

where yim corresponds to data for observation i and group m, n is the total number of observations, and b0m and εim are independent of each other. After substituting the group-level parameters in the first-level model, the model for the response vector becomes

${y}_{im}=\underset{fixed\text{\hspace{0.17em}}effects}{\underbrace{{\beta }_{00}+{\beta }_{1}{x}_{im}}}+\underset{random\text{\hspace{0.17em}}effects}{\underbrace{{b}_{0m}}}+{\epsilon }_{im}.$

A random intercept and slope model with one continuous predictor variable x, where both the intercept and slope vary independently by a grouping variable with M levels is

$\begin{array}{l}{y}_{im}={\beta }_{0m}+{\beta }_{1m}{x}_{im}+{\epsilon }_{im},\text{ }i=1,2,...,n,\text{ }m=1,2,...,M,\text{ }{\epsilon }_{im}~N\left(0,{\sigma }^{2}\right),\\ {\beta }_{0m}={\beta }_{00}+{b}_{0m},\text{ }{b}_{0m}~N\left(0,{\sigma }_{0}^{2}\right),\text{ }\\ {\beta }_{1m}={\beta }_{10}+{b}_{1m},\text{ }{b}_{1m}~N\left(0,{\sigma }_{1}^{2}\right),\end{array}$

or

${b}_{m}=\left(\begin{array}{l}{b}_{0m}\\ {b}_{1m}\end{array}\right)~N\left(0,\left(\begin{array}{cc}{\sigma }_{0}^{2}& 0\\ 0& {\sigma }_{1}^{2}\end{array}\right)\right).$

You might also have correlated random effects. In general, for a model with a random intercept and slope, the distribution of the random effects is

${b}_{m}=\left(\begin{array}{l}{b}_{0m}\\ {b}_{1m}\end{array}\right)~N\left(0,\sigma {}^{2}D\left(\theta \right)\right),$

where D is a 2-by-2 symmetric and positive semidefinite matrix, parameterized by a variance component vector θ.

After substituting the group-level parameters in the first-level model, the model for the response vector is

${y}_{im}=\underset{fixed\text{\hspace{0.17em}}effects}{\underbrace{{\beta }_{00}+{\beta }_{10}{x}_{im}}}+\underset{random\text{\hspace{0.17em}}effects}{\underbrace{{b}_{0m}+{b}_{1m}{x}_{im}}}+{\epsilon }_{im},\text{ }i=1,2,...,n,\text{ }m=1,2,...,M.$

If you express the group-level variable, xim, in the random-effects term by zim, this model is

${y}_{im}=\underset{fixed\text{\hspace{0.17em}}effects}{\underbrace{{\beta }_{00}+{\beta }_{10}{x}_{im}}}+\underset{random\text{\hspace{0.17em}}effects}{\underbrace{{b}_{0m}+{b}_{1m}{z}_{im}}}+{\epsilon }_{im},\text{ }i=1,2,...,n,\text{ }m=1,2,...,M.$

In this case, the same terms appear in both the fixed-effects design matrix and random-effects design matrix. Each zim and xim correspond to the level m of the grouping variable.

It is also possible to explain more of the group-level variations by adding more group-level predictor variables. A random-intercept and random-slope model with one continuous predictor variable x, where both the intercept and slope vary independently by a grouping variable with M levels, and one group-level predictor variable vm is

$\begin{array}{l}{y}_{im}={\beta }_{0im}+{\beta }_{1im}{x}_{im}+{\epsilon }_{im},\text{ }i=1,2,...,n,\text{ }m=1,2,...,M,\text{ }{\epsilon }_{im}~N\left(0,{\sigma }^{2}\right),\\ {\beta }_{0im}={\beta }_{00}+{\beta }_{01}{v}_{im}+{b}_{0m},\text{ }{b}_{0m}~N\left(0,{\sigma }_{0}^{2}\right),\text{ }\\ {\beta }_{1im}={\beta }_{10}+{\beta }_{11}{v}_{im}+{b}_{1m},\text{ }{b}_{1m}~N\left(0,{\sigma }_{1}^{2}\right).\end{array}$

This model results in main effects of the group-level predictor and an interaction term between the first-level and group-level predictor variables in the model for the response variable as

$\begin{array}{l}{y}_{im}={\beta }_{00}+{\beta }_{01}{v}_{im}+{b}_{0m}+\left({\beta }_{10}+{\beta }_{11}{v}_{im}+{b}_{1m}\right){x}_{im}+{\epsilon }_{im},\text{ }i=1,2,...,n,\text{ }m=1,2,...,M,\\ \text{ }\text{ }=\underset{fixed\text{\hspace{0.17em}}effects}{\underbrace{{\beta }_{00}+{\beta }_{10}{x}_{im}+{\beta }_{01}{v}_{im}+{\beta }_{11}{v}_{im}{x}_{im}}}+\underset{random\text{\hspace{0.17em}}effects}{\underbrace{{b}_{0m}+{b}_{1m}{x}_{im}}}+{\epsilon }_{im}.\end{array}$

The term β11vmxim is often called a cross-level interaction in many textbooks on multilevel models. The model for the response variable y can be expressed as

$\begin{array}{l}{y}_{im}=\left[\begin{array}{cccc}1& {x}_{1}{}_{im}& {v}_{im}& {v}_{im}{x}_{1im}\end{array}\right]\left[\begin{array}{c}{\beta }_{00}\\ {\beta }_{10}\\ {\beta }_{01}\\ {\beta }_{11}\end{array}\right]+\left[\begin{array}{cc}1& {x}_{1im}\end{array}\right]\left[\begin{array}{c}{b}_{0m}\\ {b}_{1m}\end{array}\right]+{\epsilon }_{im},\text{ }i=1,2,...,n,\text{ }m=1,2,...,M,\\ \text{ }\text{ }\end{array}$

which corresponds to the standard form given earlier,

$\text{ }y=X\beta +Zb+\epsilon .$

In general, if there are R grouping variables, and m(r,i) shows the level of grouping variable r, for observation i, then the model for the response variable for observation i is

${y}_{i}={x}_{i}^{T}\beta +\sum _{r=1}^{R}{z}_{ir}{b}_{m\left(r,i\right)}^{\left(r\right)}+{\epsilon }_{i},\text{ }i=1,2,...,n,$

where β is a p-by-1 fixed-effects vector, b(r)m(r,i) is a q(r)-by-1 random-effects vector for the rth grouping variable and level m(r,i), and εi is a 1-by-1 error term for observation i.

## References

[1] Pinherio, J. C., and D. M. Bates. Mixed-Effects Models in S and S-PLUS. Statistics and Computing Series, Springer, 2004.

[2] Hariharan, S. and J. H. Rogers. “Estimation Procedures for Hierarchical Linear Models.” Multilevel Modeling of Educational Data (A. A. Connell and D. B. McCoach, eds.). Charlotte, NC: Information Age Publishing, Inc., 2008.

[3] Hox, J. Multilevel Analysis, Techniques and Applications. Lawrence Erlbaum Associates, Inc., 2002

[4] Snidjers, T. and R. Bosker. Multilevel Analysis. Thousand Oaks, CA: Sage Publications, 1999.

[5] Gelman, A. and J. Hill. Data Analysis Using Regression and Multilevel/Hierarchical Models. New York, NY: Cambridge University Press, 2007.
{"extraction_info": {"found_math": true, "script_math_tex": 0, "script_math_asciimath": 0, "math_annotations": 0, "math_alttext": 0, "mathml": 14, "mathjax_tag": 0, "mathjax_inline_tex": 0, "mathjax_display_tex": 0, "mathjax_asciimath": 0, "img_math": 0, "codecogs_latex": 0, "wp_latex": 0, "mimetex.cgi": 0, "/images/math/codecogs": 0, "mathtex.cgi": 0, "katex": 0, "math-container": 0, "wp-katex-eq": 0, "align": 0, "equation": 0, "x-ck12": 0, "texerror": 0, "math_score": 0.8673076629638672, "perplexity": 2129.312825549007}, "config": {"markdown_headings": true, "markdown_code": false, "boilerplate_config": {"ratio_threshold": 0.18, "absolute_threshold": 10, "end_threshold": 15, "enable": true}, "remove_buttons": true, "remove_image_figures": true, "remove_link_clusters": true, "table_config": {"min_rows": 2, "min_cols": 3, "format": "plain"}, "remove_chinese": true, "remove_edit_buttons": true, "extract_latex": true}, "warc_path": "s3://commoncrawl/crawl-data/CC-MAIN-2019-26/segments/1560628000894.72/warc/CC-MAIN-20190627055431-20190627081431-00127.warc.gz"}