Normal equation for linear regression is illogical
Hello Oleksii and welcome to DSSE.
The formula you are asking about is not for a pseudoinverse.
$\theta = (X^TX)^{-1}X^Ty$
Where

$\theta$ is your regressor
$X$ is a matriz containing stacked vectors (as rows) of your features/independent variables
$y$ is a matriz containing stacked vectors (or scalars) of your predictions/dependent variables

This equation is a solution to a linear set of equations:
$Ax = B$ that occurs in trying to minimize the least squares loss.
The reason why you see this pinv() on the code is because if X has not enough linearly independent rows, $X^TX$ (also known as $R$, the autocorrelation matriz of the data, and it's inverse is called the precision matriz) will result in a singular (or near singular) matriz, which inversion might not be possible. Even if this singularity only happens because of the working precision of your computer/programming language.
Using pinv() is usually not recommended, because even if it allows to compute a regressor it will overfit the training data. Alternative solutions to working with singular matriz is adding $\delta~I$ to $R$
where $\delta$ is a small constant (usually 1 to 10 eps) and $I$ is the identity.