# Generalized inverse

#### 2022-12-08

In matrix algebra, the inverse of a matrix is defined only for square matrices, and if a matrix is singular, it does not have an inverse.

The generalized inverse (or pseudoinverse) is an extension of the idea of a matrix inverse, which has some but not all the properties of an ordinary inverse.

A common use of the pseudoinverse is to compute a ‘best fit’ (least squares) solution to a system of linear equations that lacks a unique solution.

library(matlib)

Construct a square, singular matrix [See: Timm, EX. 1.7.3]

A <-matrix(c(4, 4, -2,
4, 4, -2,
-2, -2, 10), nrow=3, ncol=3, byrow=TRUE)
det(A)
##  0

The rank is 2, so inv(A) won’t work

R(A)
##  2

In the echelon form, this rank deficiency appears as the final row of zeros

echelon(A)
##      [,1] [,2] [,3]
## [1,]    1    1    0
## [2,]    0    0    1
## [3,]    0    0    0

inv() will throw an error

try(inv(A))
## Error in Inverse(X, tol = sqrt(.Machine\$double.eps), ...) :
##   X is numerically singular

A generalized inverse does exist for any matrix, but unlike the ordinary inverse, the generalized inverse is not unique, in the sense that there are various ways of defining a generalized inverse with various inverse-like properties. The function matlib::Ginv() calculates a Moore-Penrose generalized inverse.

(AI <- Ginv(A))
##         [,1] [,2]    [,3]
## [1,] 0.27778    0 0.05556
## [2,] 0.00000    0 0.00000
## [3,] 0.05556    0 0.11111

We can also view this as fractions:

Ginv(A, fractions=TRUE)
##      [,1] [,2] [,3]
## [1,] 5/18    0 1/18
## [2,]    0    0    0
## [3,] 1/18    0  1/9

### Properties of generalized inverse (Moore-Penrose inverse)

The generalized inverse is defined as the matrix $$A^-$$ such that

• $$A * A^- * A = A$$ and
• $$A^- * A * A^- = A^-$$
A %*% AI %*% A
##      [,1] [,2] [,3]
## [1,]    4    4   -2
## [2,]    4    4   -2
## [3,]   -2   -2   10
AI %*% A %*% AI
##         [,1] [,2]    [,3]
## [1,] 0.27778    0 0.05556
## [2,] 0.00000    0 0.00000
## [3,] 0.05556    0 0.11111

In addition, both $$A * A^-$$ and $$A^- * A$$ are symmetric, but neither product gives an identity matrix, A %*% AI != AI %*% A != I

zapsmall(A %*% AI)
##      [,1] [,2] [,3]
## [1,]    1    0    0
## [2,]    1    0    0
## [3,]    0    0    1
zapsmall(AI %*% A)
##      [,1] [,2] [,3]
## [1,]    1    1    0
## [2,]    0    0    0
## [3,]    0    0    1

## Rectangular matrices

For a rectangular matrix, $$A^- = (A^{T} A)^{-1} A^{T}$$ is the generalized inverse of $$A$$ if $$(A^{T} A)^-$$ is the ginv of $$(A^{T} A)$$ [See: Timm: EX 1.6.11]

A <- cbind( 1, matrix(c(1, 0, 1, 0, 0, 1, 0, 1), nrow=4, byrow=TRUE))
A
##      [,1] [,2] [,3]
## [1,]    1    1    0
## [2,]    1    1    0
## [3,]    1    0    1
## [4,]    1    0    1

This $$4 \times 3$$ matrix is not of full rank, because columns 2 and 3 sum to column 1.

R(A)
##  2
(AA <- t(A) %*% A)
##      [,1] [,2] [,3]
## [1,]    4    2    2
## [2,]    2    2    0
## [3,]    2    0    2
(AAI <- Ginv(AA))
##      [,1] [,2] [,3]
## [1,]  0.5 -0.5    0
## [2,] -0.5  1.0    0
## [3,]  0.0  0.0    0

The generalized inverse of $$A$$ is $$(A^{T} A)^- A^{T}$$, AAI * t(A)

AI <- AAI  %*%  t(A)

Show that it is a generalized inverse:

A %*% AI %*% A
##      [,1] [,2] [,3]
## [1,]    1    1    0
## [2,]    1    1    0
## [3,]    1    0    1
## [4,]    1    0    1
AI %*% A %*% AI
##      [,1] [,2] [,3] [,4]
## [1,]  0.0  0.0  0.5  0.5
## [2,]  0.5  0.5 -0.5 -0.5
## [3,]  0.0  0.0  0.0  0.0
{"ft_lang_label":"__label__en","ft_lang_prob":0.6658291,"math_prob":0.99921006,"size":2944,"snap":"2023-14-2023-23","text_gpt3_token_len":1354,"char_repetition_ratio":0.18639456,"word_repetition_ratio":0.21428572,"special_character_ratio":0.54313856,"punctuation_ratio":0.22107969,"nsfw_num_words":0,"has_unicode_error":false,"math_prob_llama3":0.99300736,"pos_list":[0],"im_url_duplicate_count":[null],"WARC_HEADER":"{\"WARC-Type\":\"response\",\"WARC-Date\":\"2023-06-06T22:56:25Z\",\"WARC-Record-ID\":\"<urn:uuid:e4967f4d-d681-43c6-850a-2a1456143515>\",\"Content-Length\":\"19444\",\"Content-Type\":\"application/http; msgtype=response\",\"WARC-Warcinfo-ID\":\"<urn:uuid:205f603c-0c85-46f4-a370-a7aa3d59f25a>\",\"WARC-Concurrent-To\":\"<urn:uuid:cbf2d0db-ecb9-4e09-b97e-d9cab4d7a29a>\",\"WARC-IP-Address\":\"128.176.130.197\",\"WARC-Target-URI\":\"https://cran.uni-muenster.de:443/web/packages/matlib/vignettes/ginv.html\",\"WARC-Payload-Digest\":\"sha1:IUYI5VI2INCPXRG6F4JQDYUL4XPUJTJA\",\"WARC-Block-Digest\":\"sha1:4DUG4LAJG7FLHOR6QVW33DDMAZLKWTHE\",\"WARC-Identified-Payload-Type\":\"text/html\",\"warc_filename\":\"/cc_download/warc_2023/CC-MAIN-2023-23/CC-MAIN-2023-23_segments_1685224653183.5_warc_CC-MAIN-20230606214755-20230607004755-00753.warc.gz\"}"}