\section{Introduction}

Finding the casual relationship  among  several  stochastic  series of signals corresponding to $N$ sources represents a formidable challenge. The role of the statistical analysis  for this task was already noticed by Masani and Wiener in the sixties of the XX century~\cite{WIENERMASANI}, to be followed by Granger~\cite{GRANGER}  and developed  by many~\cite{MANY,MANY2,MANY3,MANY4,MANY5,MANY6}.  In the multivariate analysis, the study of the cross-correlations is  perhaps  the most common method. Historically, Wishart~\cite{WISHART} was the first to ask  how to generalize the chi-squared distribution for the case of multiple dimensions, corresponding to the matrix of Pearson correlation coefficients of purely random time series.
Wishart ensemble may be also the first random matrix application in science. If we consider multivariate time series represented by a matrix $X_{it}$, where the 'space' index takes values from $\{1, \dots, N\}$,
and 'time' index  $t\in\{1, \dots, T\}$, the empirical correlation matrix  can be represented as $C_{ij}=\frac{1}{T} \sum_{t=1}^T x_{it}\bar{x}_{jt}$ or, in a matrix notation,
$\bb{C}=\frac{1}{T}\bb{xx}^{\dagger}$, where  bar ($\dagger$, respectively) applies to the more general case  of the data valued in complex numbers. Lowercase $x$ denotes  the standardized time series, i.e. after a procedure  when for each individual time series we subtract the corresponding mean and divide the result be the corresponding variance.
Wishart ensemble corresponds to a maximally random correlation matrix, where
each entry $x_{it}$ of a rectangular signal matrix is drawn from a real (complex) Gaussian distribution, so
$\<x_{it} \bar{x}_{jt^{'}}\>=\delta_{ij}\delta_{tt^{'}}$, where angle brackets  represent taking the expectation values with respects to the probabilistic measure (here Gaussian).  In general, the time series may include
correlations. In particular, assuming space and time factorization,  $\<x_{it} \bar{x}_{jt^{'}}\>=A_{ij}B_{tt^{'}}$ where $\bb{A}$ and $\bb{B}$ are symmetric (Hermitian) positive definite matrices. The spectral properties of such  single-correlated  ($\bb{A}=\One_N $ or $\bb{B}=\One_T$) \cite{WISHSINGLE,WISHSINGLE1,WISHSINGLE2,WISHSINGLE3} or doubly-correlated ($\bb{A} \neq \One_N $ and  $\bb{B} \neq \One_T$) \cite{WISHDOUBLY,BURDAJURKIEWICZ}
Wishart ensembles were extensively studied in the literature.

When the number of consecutive measurements (the length of the series) tends to infinity the empirical correlation matrix tends to the true correlation matrix. Unfortunately, there are systems in which one cannot repeat measurements due to uniqueness of data (e.g. study of climate) or the fact that the number of independent time series is of the same order as the number of measurements (e.g. stock markets). In such instances the empirical correlation matrix deviates from the true correlation matrix. The Wishart matrix corresponding to such situation can be obtained in a limiting procedure $N,T\to\infty$ with $r=N/T$ fixed. This limit is referred to as Random Matrix Theory (RMT) limit or Big Data limit~\cite{BOUCHAUDREVIEW}. The parameter $r$, pertaining to the rectangularity of the array of collected data, is also known as the signal-to-noise ratio. Even having the ability to collect a vast number of data points, one would like to extract the leading portion of information from the system. Such a procedure relies on the diagonalization of the correlation matrix and focusing on the largest eigenvalues and corresponding eigenvectors and bears the name of Principal Component Analysis (PCA).

The Wishart ensemble has found applications in telecommunication~\cite{TELECOM} and quantum information~\cite{QUANTUM}.
The applications of PCA in the Big Data limit range from financial  engineering ~\cite{FIN1,FIN2}, through  genetics~\cite{GENETICS,GENETICS2}, meteorology and oceanography~\cite{METEOROLOGY}, study of atmosphere~\cite{ATMO}, climate change detection~\cite{CLIMATE}, criminal offence records~\cite{CRIMINAL}, to the analysis of EEG data~\cite{EEG}.

The analysis of correlation matrices presented above gives an insight into the interdependence of the components of complex systems \textit{at} the same moments. In order to gain the access to the correlations at different instants, one has to shift the time index when correlating the time series, obtaining the auto cross-correlation matrix $C_{ij}^{\tau}=\frac{1}{T-\tau}\sum_{t=1}^{T-\tau}x_{it}\bar{x}_{j,t+\tau}$, which explicitly depends on the time lag $\tau$. Such object appeared in the literature also under the name of delayed or time-lagged correlation matrix. For simplicity, we refer to this as the lagged correlation matrix. By construction $\bb{C}^{\tau}$ is not symmetric, which reflects the asymmetric influence of the component of the system. Such an information is not accessible by means of equal time correlation matrices.

The analysis of time-lagged correlations is recently becoming an area of a rapid development~\cite{LAG1,LAG2,LIVAN,LAG3,LAG4,LAG5}. The diagonalization of the asymmetric lagged correlation matrices in the financial context was proposed in \cite{DROZDZ} and in studies of human brain activity in \cite{DROZDZ2}. Later on, Podobnik et al. studied~\cite{PODOBNIK} the singular values of the lagged correlation matrices in various complex systems, including EEG signal and stock market, finding the long-range magnitude correlations.

Because of the asymmetry of the lagged correlation matrix, its spectrum is complex, which invalidates the standard tools of random matrix theories. Finding the RMT benchmark of the spectrum, corresponding to the absence of any correlations within the system is a formidable challenge. Due to the importance of the lagged correlations, this spectral problem was attacked few times in the literature, using the tools of RMT: \\
(i) \textit{Symmetrization.} First, the symmetrized  lagged covariance matrix $\bb{C}^{{\rm sym}}=\frac{1}{2}\left(\bb{C}^{\tau}+(\bb{C}^{\tau})^{\dagger}\right)$ was studied~\cite{HINDU,USREVIEW}.  In the Big Data limit ($N,T \rightarrow \infty$ with $N/T$ fixed) the  resulting real spectral density comes from the resolvent (Green's function) fulfilling a quartic algebraic  equation (Ferrari equation).\\
 (ii) \textit{Spectral whitening.} The singular value decomposition of the product $(\bb{C}^{\tau})^{\dagger} \bb{C}^{\tau}$ was studied, after removing all equal-time correlations in order to define maximally random product~\cite{BOUCHAUD}. In the RMT limit, the resulting spectrum is  given by the so-called free Jacobi measure.\\
(iii) \textit{Abelization.} Third,  the strictly non-Hermitian lagged correlation matrix problem was approached in ~\cite{THURNERBIELY}, using  the inverse Abel transform for a circularly symmetric spectrum.   In this case, the radial spectral function stems from the quartic algebraic equation from the approach (i). We call this method 'Abelization', as it refers to the Abel transform and works only for matrices that commute with their Hermitian conjugate.\\
(iv) Finally, the explicit non-Hermitian lagged correlation matrices were analized by Jarosz~\cite{JAROSZLAG} using the non-Hermitian diagrammatic techniques. In particular, the result (iii) was challenged -  the radial spectral  function  was originating from the cubic algebraic equation (Cardano type).\\
(v) Later on, Livan and Rebecchi~\cite{LIVAN} proposed another approach relying on the assumption that the benchmark for the lagged correlation matrix can be approximated by a product of two independent rectangular Gaussian matrices, the spectral density of which was calculated in~\cite{BJLNSproducts} and is given by the solution of a quadratic equation.

In this paper, we first summarize the above approaches using the powerful version of Random Matrix Theory in the large size limit, called  free random variables.
This technique represents a remarkable short-cut for calculations of various correlation matrices, in the limit when the number of measured components $N$ and the number of measurements $T$ are large, but their ratio $r=N/T$ is finite~\cite{USREVIEW,BOUCHAUDLACES}.
Section~\ref{sec:FRV} summarizes the basic operational tools of this technique. Section~\ref{sec:FRV2}  shows its applications in the approaches (i)-(iv).  In particular, we resolve the controversy between~\cite{THURNERBIELY} and ~\cite{JAROSZLAG}, in favor of~\cite{JAROSZLAG}. This is the first  main result of  this paper.

In section~\ref{sec:diagrammatic}, we recall a certain linearization method for study the non-Hermitian random matrix ensembles. We present the diagrammatic construction in the asymptotic limit. To find the spectrum of the lagged correlation matrix, we solve a more general problem, namely the spectrum of the matrix $\bb{Y}=\frac{1}{T}\bb{XAX}^{\dagger}$, where $\bb{X}$ is an $N\times T$ Gaussian random matrix and $\bb{A}$ is \textit{any} matrix, indepenent of $\bb{X}$, not necessarily symmetric (Hermitian). As a next new result, we also present the  left-  right-eigenvectors correlations for such a problem. To the best of our knowledge, such analysis has never been done before.
Then, in section~\ref{sec:Analysis}, we apply this result to the particular instance of the lagged correlation matrices also in a double scaling limit $N,T,\tau\to\infty$ with $r=N/T$ {\it and} $\tau/T$ fixed and compare  with the numerical simulation.  Section~\ref{sec:Conclusions} concludes the paper and outlines the further prospects of this method.  To facilitate the reading of the manuscript, we relegated several technical details and the thorough analysis of Feynman diagrams to the appendices.

\textit{Notation}. In order to avoid ambiguity we adopt a notation at which matrices of a standard size are written with a boldface font while its elements in a standard way. In the sections devoted to the non-Hermitian matrices we denote the matrices with a doubled block structure by the calligraphic letters.

\section{Addition and multiplication theorems for random ensembles} \label{sec:FRV}
\subsection{Real spectra}
In the classical probability theory, the problem of finding the probability distribution function (pdf) $p_{1+2}(s) $    of a sum $s$ of two independent random variables $x$ and $y$ resulting from the corresponding  measures  $p_1(x)dx$ and $p_2(y)dy$ is solved with the help of the characteristic function.  The characteristic function, which is in fact the Fourier  transform of the pdf, factorizes the convolution problem $p(s)=\int dx dy p_1(x)p_2(y) \delta(s-(x+y))$ into a product of the individual characteristic functions for both pdfs.  Since the characteristic function is the generating function for the moments of $p_{1+2}(x)$,  the calculation of moments of the sum boils down to combinatorics. Further simplification comes from considering the logarithm of the characteristic function - the multiplication of the characteristic functions is then replaced   by the addition of their logarithms. This sum rule  holds for each  term in the series expansion,   so the corresponding coefficients, called cumulants, are additive under convolution.
Similarly, the Mellin transform  reduces the problem of finding probability distribution for the product $z=xy$ of the aforementioned variables to the simple multiplication of the corresponding problems~\cite{EPSTEIN}.

Free random variable (FRV) calculus provides tools, which superimpose the above addition and multiplication laws in the case, when variables $x$ and $y$ are replaced by  asymptotically infinite, non-commuting matrices $\bb{X}$ and $\bb{Y}$. One asks what the spectral measure of $\bb{X}+\bb{Y}$ and $\bb{XY}$ is, provided that the individual spectral measures for both $\bb{X}$ and $\bb{Y}$ are known. As such, FRV represents a certain non-commutative counterpart  of a classical probability calculus, ideally suited for the multivariate analysis of large matrices.
The main objects in FRV are defined as follows.
We are interested in finding the distribution of the {\it real} eigenvalues
$\lambda_i $ of some stochastic matrix $\bb{H}$  in the limit when  $N$ (size of the matrix) tends to infinity.  It is convenient to introduce the {\it complex} traced resolvent (Green's function)
\begin{eqnarray} G_\bb{H}(z)=\frac{1}{N} \left\langle {\rm Tr}\,
\frac{1}{
z\One_N-\bb{H}}\right\rangle \,. \label{green}
\end{eqnarray}}\def\no{\nonumber
where the brackets $\<...\>$ represent the averaging over the ensemble of $N \times
N$ random Hermitian matrices generated from the probability distribution function
 \begin{eqnarray} P(\bb{H})
\propto e^{-N {\rm Tr} V(\bb{H})} . \label{probab} \end{eqnarray}}\def\no{\nonumber
 On the basis of the Sochocki-Plemelj formula
 \begin{equation}
 \lim_{\epsilon\to 0} \frac{1}{x \pm i \epsilon} = P.V. \frac{1}{x} \mp i  \pi \delta (x),
 \end{equation}
  the imaginary part of $G(z)$  reconstructs the desired spectral density,
\begin{eqnarray}
-\frac{1}{\pi} \lim_{\epsilon \rightarrow 0^+} G(z)|_{z=\lambda+i\epsilon}=\< \frac{1}{N} \sum\limits_{i=1}^{N} \delta (\lambda -\lambda_i) \> \equiv  \rho(\lambda).
\end{eqnarray}}\def\no{\nonumber
The expansion of the  Green's function for large $z$ generates all spectral moments  $\mu_n=\frac{1}{N} \left \langle {\rm Tr} \bb{H}^n \right \rangle$, alike the characteristic function generates the moments of the pdf in classical probability calculus
\begin{equation}
G(z)=\sum_{k=1}^{\infty}\frac{1}{z^{k+1}} \<\frac{1}{N}\mbox{Tr} \bb{H}^k\>. \label{eq:HermMoments}
\end{equation}
  Motivated by the concepts of the classical probability calculus, Voiculescu~\cite{VOICULESCU} introduced the
R-transform as the generating function for cumulants
\begin{eqnarray}
R(z)=
\sum_{n=1}^{\infty} \kappa_n z^{n-1},
\label{Rdef}
\end{eqnarray}}\def\no{\nonumber
i.e. $R(z)$ is the analogue of the logarithm of the characteristic function in probability theory.
Both complex functions $G(z)$ and $R(z)$ are related via
\begin{eqnarray}
G\left[R(z)+\frac{1}{z}\right]=R[G(z)] +\frac{1}{G(z)}=z\, .
\label{Rtransform}
\end{eqnarray}}\def\no{\nonumber
As an example we consider the instance, which is an analogue of the centered Gaussian, i.e. its distribution is completely determined by the second cumulant only, since all other vanish. Its FRV analogue is therefore $R_s(z)=\kappa_2 z$, which by (\ref{Rtransform}) leads to the quadratic equation for the Green's function with solutions
\begin{eqnarray}
G_s(z)=\frac{1}{2\kappa_2 z} \left(z \mp \sqrt{z^2-4 \kappa_2}\right).
\label{WignerGreen}
\end{eqnarray}}\def\no{\nonumber
For large $z$, only the  negative sign provides the correct asymptotic  limit $G_s(z) \sim \frac{1}{z}$. Taking the imaginary part according to  (\ref{green}), we arrive at the celebrated  Wigner semicircle distribution  of Random Matrix Theory
\begin{eqnarray}
\rho_s(\lambda)=\frac{1}{2\pi \kappa_2} \sqrt{ 4\kappa_2-\lambda^2}.
\end{eqnarray}}\def\no{\nonumber
The Wigner distribution is therefore the FRV counterpart of the Gaussian distribution in the classical probability theory.

The crucial notion in free probability  is the freeness condition which replaces independence in the classical probability calculus.
The most intuitive definition of freeness of $\bb{A}$ and $\bb{B}$ is based on the complete decorrelation of the corresponding  eigenvectors, i.e. their eigenbases are maximally random oriented. The pair $\bb{A}$ and $\bb{UBU}^{\dagger}$ becomes free in the limit when the size of matrices tends to infinity, provided  that $\bb{U}$ is Haar unitary (i.e. infinitely large random unitary matrix from Circular Unitary Ensemble).  For a free pair $(\bb{A},\bb{B})$ a powerful addition law holds
\begin{eqnarray}
R_{\bb{A}+\bb{B}}(z)=R_{\bb{A}}(z)+R_{\bb{B}}(z),
\label{hermadd}
\end{eqnarray}}\def\no{\nonumber
linearizing the spectral problem of convolution of two non-commuting free random ensembles.
Since the R-transforms are the generating functions for the free cumulants, the above law imposes the additivity of free cumulants, in an analogy to the addition law for the logarithms  of the characteristic functions in the classical probability theory.

It is suitable here to demonstrate the difference between the classical and FRV calculus on the simple example of convolution of two distributions, each of them composed  of a binary set of -1/2 and 1/2.  In the classical case, the resulting distribution is discrete,  where the values -1 and  1 are obtained with the probability 1/4 each, and the value 0 is obtained with the probability 1/2. The matrix-valued  version of this example could be realized in terms of two identical large matrices $\bb{A}$ and $\bb{B}$, each of them having on the diagonal the equal number of -1/2 and 1/2.  The spectral density functions are therefore given by
\begin{eqnarray}
\rho_\bb{A}(\lambda)=\rho_\bb{B}(\lambda)=\frac{1}{2}\delta\left(\lambda+\frac{1}{2}\right)+\frac{1}{2}\delta\left(\lambda- \frac{1}{2}\right)
\label{projectors}
\end{eqnarray}}\def\no{\nonumber
or, equivalently, the Green's functions read
\begin{eqnarray}
G_\bb{A}(z)=G_\bb{B}(z)=\frac{1}{2} \left(  \frac{1}{z+\frac{1}{2}} +\frac{1}{z-\frac{1}{2}} \right).
\end{eqnarray}}\def\no{\nonumber
Substituting $ z \rightarrow R(G_{\bb{A}}(z)) + 1/G_{\bb{A}}(z)$ and using the definition (\ref{Rtransform}), one arrives first at $R_{\bb{A}}(z)= \frac{\sqrt{1+z^2}-1}{2z}=R_{\bb{B}}(z)$.  Then, making use of the addition law,  $R_{\bb{A}+\bb{B}}(z)=R_{\bb{A}}(z)+R_{\bb{B}}(z)$ and using \eqref{Rtransform} but in the opposite direction, one finally arrives at
\begin{eqnarray}
G_{\bb{A}+\bb{B}}(z)=\frac{1}{\sqrt{z^2-1}},
\label{Greenarcsin}
\end{eqnarray}}\def\no{\nonumber
which, via the imaginary part, leads to the {\it continuous} spectral arsine law
\begin{eqnarray}
\rho_{\bb{A}+\bb{B}}(\lambda)=\frac{1}{2 \pi \sqrt{(1-\lambda)(1+\lambda)}}.
\label{arcsine}
\end{eqnarray}}\def\no{\nonumber
One can easily check this numerically by adding large matrices $\bb{A}$ and $\bb{UBU}^{\dagger}$, where $\bb{U}$ comes from the Haar measure.  Diagonalizing the sum and plotting the histogram of the resulting eigenvalues, one obtains the spectral distribution \eqref{arcsine}.

Surprisingly, a similar construction exists for the product of ensembles $\bb{A}$ and $\bb{B}$, despite in general the product of Hermitian (symmetric)  operators is not necessarily Hermitian.  If one of the matrices is positive (e.g. $\bb{A}$), the moments $\mbox{Tr}(\bb{AB})^k$ can be rewritten under trace as $\mbox{Tr}(\sqrt{\bb{A}}\bb{B}\sqrt{\bb{A}})^k$, while $\sqrt{\bb{A}}\bb{B}\sqrt{\bb{A}}$ is Hermitian by construction and shares the same eigenvalues as $\bb{AB}$. This observation leads to  the  corresponding multiplication law~\cite{BJNPRODUCTS}
\begin{eqnarray}
R_{\bb{AB}}(G_{\bb{AB}})=R_\bb{A}(G_\bb{B})R_\bb{B}(G_\bb{A}),
\label{hermmult}
\end{eqnarray}}\def\no{\nonumber
where
\begin{eqnarray}
G_{\bb{A}}=G_{\bb{AB}}R_\bb{A}(G_{\bb{B}}), \,\,\,\,\,\,\,\,\,\,\,\,\,\,  G_\bb{B}=G_{\bb{AB}}R_\bb{B}(G_\bb{A}).
\label{hermmult1}
\end{eqnarray}}\def\no{\nonumber
The original construction of the multiplication law by  Voiculescu~\cite{VOICULESCU} (S-transform) is related to this formulation via
\begin{eqnarray}
R(z)=\frac{1}{S(zR(z))}, \,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\, S(z)=\frac{1}{R(zS(z))}, \label{Stransform}
\end{eqnarray}}\def\no{\nonumber
so that $S_{\bb{AB}}(z)=S_\bb{A}(z)S_\bb{B}(z)$. Note that the formulation in the language of the S-transform  requires $R(0) \neq 0$ for both ensembles $\bb{A}$ and $\bb{B}$, which may be weakened~\cite{SPEICHERRAO}. Equations (\ref{hermmult},\ref{hermmult1}) are still valid, even if $R(0)=0$.

Finally, let us consider the important case when the resulting non-Hermitian matrix $\bb{X}$ can be decomposed as $\bb{X}=\bb{PU}$, where $\bb{P}$ is positive, $\bb{U}$ is Haar unitary and $\bb{P}$ and $\bb{U}$ are free. In such a case, the complex spectrum possesses a polar symmetry, so the spectral problem is quasi-one dimensional, i.e. only the radial spectral density $\rho(\lambda,\bar{\lambda})=\rho(|\lambda| = s)$ is non-trivial. In such circumstances the  Haagerup-Larsen theorem provides a remarkably simple relation between the radial cumulative distribution function $F(s)=2\pi \int_0^s  s' \rho(s') ds'$ and the S-transform of $\bb{P}^2$~\cite{HAAGERUPLARSEN}
\begin{eqnarray}
S_{\bb{P}^2}(F(s)-1)=\frac{1}{s^2}. \label{eq:HaagLars}
\end{eqnarray}}\def\no{\nonumber
The spectrum is always confined to the ring with radii $s_{min}$,  $s_{max}$, where
$s_{min}^{-2}=\int_{0}^{\infty}x^{-2} \rho_{\bb{P}}(x)dx$ and $s_{max}^2=\int_{0}^{\infty}x^2\rho_{\bb{P}}(x)dx$. Note that this case includes  in particular $s_{min}=0$ and $s_{max}=\infty$. This is the so-called single ring theorem~\cite{FEINBERGZEE}. Recently, it was proven~\cite{HAAGLARSVECTORS}, that the same radial cumulative distribution function $F(s)$  provides the information about certain eigenvector correlator
\begin{eqnarray}
O(s) \equiv \lim_{N \rightarrow \infty} \frac{1}{N^2} \left< \sum_{\alpha} O_{\alpha \alpha} \delta^{(2)} (\lambda-\lambda_{\alpha})\right>=\frac{F(s)(1-F(s))}{\pi s^2}, \label{eq:HaagLarsCorr}
\end{eqnarray}}\def\no{\nonumber
where  $O_{\alpha \beta} = \<L_{\alpha} |L_{\beta}\>\<R_{\beta}|R_{\alpha}\>$, and $\ket{R_{\alpha}} $ and $\bra{L_{\alpha}}$ are right and left eigenvectors of the non-Hermitian matrix $\bb{X}$, respectively (see also Sec.~ \ref{sec:NonHermitianRandomMatrices}).

\subsection{Free Random Variable cookbook}
Here we present brief recipes for the addition and multiplication of large random matrices and introduce some additional FRV transforms related to moments and cumulants. They found an application in description of spectral properties of large covariance matrices in particular in the financial context~\cite{SNARSKA,SNARSKA2}. Recently, they also turned out to be crucial in the cleaning of noisy covariance matrices~\cite{BOUCHAUDREVIEW,BOUCHAUDRIE} and studying the properties of the eigenvectors of such matrices~\cite{BOUCHVECT1}.

The procedure for addition goes as follows:
\begin{enumerate}
\item Knowing individual spectral functions $\rho_\bb{A}(\lambda)$, $\rho_\bb{B}(\lambda)$, we calculate the corresponding Green's functions $G_\bb{A}(z)$ and $G_\bb{B}(z)$.
\item Using the relation (\ref{Rtransform}), we  construct $R_\bb{A}(z)$ and $R_\bb{B}(z)$,  which we then add ({\it cf.}  (\ref{hermadd})) forming $R_{\bb{A}+\bb{B}}(z)$.
\item Finally, using again the relation (\ref{Rtransform}), we reconstruct $G_{\bb{A}+\bb{B}}(z)$, which via the imaginary part yields $\rho_{\bb{A}+\bb{B}}(\lambda)$.
\end{enumerate}
Sometimes it is practical  to use the functional inverse of the Green's function (nicknamed as 'Blue's function'), defined as
$B[G(z)]=G[B(z)]=z$ and related to the R-transform via  $B(z)=R(z)+1/z$.   The procedure for addition is identical, except the obvious shift in the addition law  $(B_{\bb{A}+\bb{B}}(z)= B_\bb{A}(z)+B_\bb{B}(z) -1/z)$.

The multiplication  algorithm is as follows:
\begin{enumerate}
\item Knowing individual spectral functions $\rho_\bb{A}(\lambda)$, $\rho_\bb{B}(\lambda)$, we write down the corresponding Green's functions $G_\bb{A}(z)$ and $G_\bb{B}(z)$, and then,  using the relation (\ref{Rtransform}), we  construct $R_\bb{A}(z)$ and $R_\bb{B}(z)$.
\item Using (\ref{Stransform}), we calculate $S_\bb{A}(z)$ and $S_\bb{B}(z)$, which we then multiply (if  the aforementioned positivity condition holds), getting
$S_{\bb{A} \cdot \bb{B}}(z)$.
\item Reversing the order of above operations, we finally reconstruct $G_{\bb{A} \cdot \bb{B}}(z)$, the imaginary part of which yields $\rho_{\bb{A} \cdot \bb{B}}(\lambda)$.
\end{enumerate}
It turns out that in some instances it is more practical to use the moment generating function  $M(z) \equiv z G(z)-1$ and its functional inverse, defined as $M[N(z)]=N[M(z)]=z$.  The procedure for the multiplication reads then $N_\bb{A}(z) N_\bb{B}(z)= \frac{1+z}{z} N_{\bb{A} \cdot \bb{B}} (z)$, since $S(z)=\frac{1+z}{z} \frac{1}{N(z)}$.

We list in  the form of the Table~\ref{tab:WishartAntiwishart} the corresponding transforms for the Wishart  $\bb{W}= \frac{1}{T} \bb{xx}^{\dagger} $ (represented by $N \times N$ matrix)  and  the anti-Wishart matrix $\bb{aW}\equiv \frac{1}{N} \bb{x}^{\dagger}\bb{x}$ (represented by $T \times T$ matrix). Note the duality relation $M_\bb{W}(z)=rM_{\bb{aW}}(rz)$.

\begin{table}[h]
\begin{center}
 \begin{tabular}{||c c c ||}
 \hline
  & Wishart  & anti-Wishart   \\
  \hline\hline

 Matrix form  & $\frac{1}{T}\bb{xx}^{\dagger}$ & $\frac{1}{N} \bb{x}^{\dagger}\bb{x} $
  \\
 \hline

 R-transform &  $R_\bb{W}(z)=\frac{1}{1-rz}$ & $R_{\bb{aW}}(z)= \frac{r}{r-z}$
  \\
 \hline

 S-transform & $S_\bb{W}(z) = \frac{1}{1+rz}$ & $S_{\bb{aW}}(z)=\frac{r}{r+z}  $
 \\
 \hline

N-transform   & $N_\bb{W}(z)=\frac{1+z}{z} (1+rz) $\,\,\,\,\,\,\,\,   & $N_{\bb{aW}}(z)=  \frac{(1+z)(r+z)}{rz}$
\\
 \hline
 \hline

\end{tabular}
\caption{Comparison between Wishart and anti-Wishart transforms. \label{tab:WishartAntiwishart}}
\end{center}
\end{table}

\section{Unit time-lagged correlation matrices --  quasi-one dimensional reductions}  \label{sec:FRV2}

We consider the Pearson estimator of the autocorrelation matrix with the time shift (lag) of $\tau$ units
\begin{eqnarray}
C_{ij}^{\tau}=\frac{1}{T-\tau}\sum_{t=1}^{T-\tau} x_{it} \bar{x}_{j,t+\tau},
\end{eqnarray}}\def\no{\nonumber
where $x_{it}$ represents measurements of $i=1,...,N$ objects at times $t=1,...,T$. First, we are interested in the unit time lag ($\tau=1$) in the limit when both $N$ and $T$ tend to infinity, keeping their ratio $r=N/T$ fixed (we choose $r \leq 1$).  Note, that in matrix notation
\begin{eqnarray}
 \bb{C}=\frac{1}{T} \bb{xDx}^{\dagger},
\end{eqnarray}}\def\no{\nonumber
with $D_{tt'}=\delta_{t+\tau,t'}$. For $T\to\infty$ and fixed $\tau$ we can put $T\sim (T-1)$. Moreover, in this section we associate $T+k$ with $k$ in the argument of the Kronecker delta in order to secure the existence of the inverse matrix for $\bb{D}$. We call this cyclic approximation, because $\bb{D}$ is then a representation of a cyclic permutation. The accuracy of such an approximation has to be checked numerically a posteriori.

Higher values of $\tau$, which we also refer to the depth of the lag, are considered later in Section \ref{sec:Analysis}, devoted to non-Hermitian matrices, where the non-trivial limiting procedure $N,T,\tau \to \infty$ with $N/T$ and $\tau/T$ fixed is studied.

\subsection{Symmetrization}
Symmetrization of the lag matrix $\bb{D}$ leads to $\bb{D}^{sym}=\frac{1}{2}(\bb{D}+\bb{D}^T)$.  The eigenvalues of the symmetrized matrix read $\lambda_k=\cos (\pi k /(T+1))$ and, in the limit $T \rightarrow \infty$, $\tau$ fixed, the resolvent can be approximated as
\begin{eqnarray}
G_{\bb{D}^{sym}}(z)=\int_0^1 dx \frac{1}{z- \cos (\pi x)}=\frac{1}{\sqrt{z^2-1}},
\end{eqnarray}}\def\no{\nonumber
 which is identical to the  resolvent for the free arcsine law \eqref{Greenarcsin}. Inserting the above result into the chain rule for the S-transforms~\cite{USREVIEW},  one obtains  the fourth order polynomial equation (Ferrari equation) for the moment generating function $M(z)$
\begin{eqnarray}
r^3 M^4 +2r^2(1+r)M^3 +r(1+4r+r^2-z^2)M^2 +2(r^2+r-z^2)M  +r=0. \label{eq:quarticM}
\end{eqnarray}}\def\no{\nonumber
This equation was first presented without a proof in \cite{HINDU}  and derived in~\cite{USREVIEW}.

\subsection{Spectral whitening}
We represent the lagged correlation matrix  as $\bb{C}^{\tau}=\frac{1}{T} \bb{xy}^{\dagger}$, where $y_{it}=x_{i,t+\tau}$. Then we calculate its 'squared modulus'
 $\bb{CC}^{\dagger}=T^{-2} \bb{xy}^{\dagger}\bb{yx}^{\dagger}$. The spectral content of such a matrix  is complicated   - the real eigenvalues not only capture the cross-correlations between $x$ and $y$, but also reflect the inner-correlation within the 'Wishart-like' ensembles  $\bb{W_x}=\frac{1}{T}\bb{xx}^{\dagger}$ and  $\bb{W_y}=\frac{1}{T} \bb{yy}^{\dagger}$. In order to disentangle these two sources of information, we remove by hand  all the correlations  within these two Wishart ensembles. We diagonalize the equal-time correlation matrix $\bb{W_x}=\bb{U \Lambda U}^{\dagger}$ and define a new time series $\bb{x}'=\bb{\Lambda}^{-1/2}\bb{U}^{\dagger}\bb{x}$,  which has the property that its equal-time correlation matrix is the identity.  This procedure, applied likewise to $\bb{y}$, is known as whitening.  The next step uses the observation that the moments of the square involve the product of two whitened anti-Wishart type ensembles $\bb{W}_1=\frac{1}{T}\bb{x}'^{\dagger}\bb{x'}$ and $\bb{W}_2=\frac{1}{T} \bb{y}'^{\dagger}\bb{y}'$.  Since both ensembles are now represented by $T \times T$ matrices, and the nonzero-eigenvalues of both Wishart and anti-Wishart ensembles are identical and equal to 1,  the remaining $T-N$ eigenvalues have to be zeroes.
 In the absence of the time-lagged correlations, the spectral problem reduces therefore to the free multiplication of two projectors, the Green's functions of which read
 $G_{\bb{W}_1}(z)=G_{\bb{W}_2}(z)=r \frac{1}{z-1}+(1-r)\frac{1}{z}$. A brief calculation (see Appendix \ref{App:FreeProjectors}) yields the quadratic equation for M
 \begin{eqnarray}
 (z-1)M^2+(z-2r)M-r^2=0.
 \end{eqnarray}}\def\no{\nonumber
 The resulting spectral density  belongs to the family of free Jacobi measures~\cite{VOICULESCU}. This method for  looking for general cross-correlations in economic data was proposed in~\cite{BOUCHAUD}.

 \subsection{Abelization}

A way to deal with the non-Hermitian matrices with circularly symmetric spectrum was proposed by Biely and Thurner \cite{THURNERBIELY} with no proof and without specifying the class of matrices which this procedure applies to. They argued that the marginal distribution $\rho_x(x)=\int\rho(z=x+iy)dy$ of such a matrix is related to the spectral density of a symmetrized problem $\bb{C}^{{\rm sym}}=\frac{1}{2}(\bb{C}^{\tau}+(\bb{C}^{\tau})^{\dagger})$ via $\rho_x(x)=\rho^{sym}(\sqrt{2}x)$. Later, it was shown \cite{BURDAJANIKWACLAW} that this property does not hold for the product of two independent GUE matrices.

We argue here that the Abelization  property holds for normal matrices $\bb{X}$, i.e. those which can be diagonalized by a unitary transformation $\bb{X}=\bb{U\Lambda U}^{\dagger}$. The Hermitian conjugate $\bb{X}^{\dagger}$ can be diagonalized by the same matrix $\bb{U}$, thus the eigenvalues of the symmetrized matrix $\bb{C}^{{\rm sym}}=\frac{1}{2}\bb{U}(\bb{\Lambda}+\bb{\Lambda}^{\dagger})\bb{U}^{\dagger}$ are the real parts of the eigenvalues of $\bb{X}$, thus $\rho_{x}(x)=\rho^{{\rm sym}}(x)$.

If the spectrum is circularly symmetric, $\rho(z,\bar{z} )=\rho(s)$, where $s=|z|$, the marginal distribution can be rewritten as the Abel transform of the spectral density:
\begin{equation}
\rho_x(x)=2\int\limits_{x}^{\infty}\frac{s \rho(s) ds}{\sqrt{s^2-x^2}}.
\end{equation}
Knowing the spectral density of the symmetrized problem, one can recover the eigenvalue spectrum of the initial matrix via the inverse Abel transform
\begin{equation}
\rho(s)=-\frac{1}{\pi}\int\limits_{s}^{\infty}\frac{d\rho^{sym}(x)}{dx}\frac{dx}{\sqrt{x^2-s^2}}.
\end{equation}
The spectral density of the symmetrized problem is obtained from \eqref{eq:quarticM}.

Unfortunately, the lagged correlation matrices are usually not normal, so a different approach has to be used to get the proper form of the radial spectral density.

\begin{figure}
\includegraphics[width=\textwidth]{fig_num1}
\caption{A numerical diagonalization of 1000 matrices of size $N=1000$ and different rectangularities. The solid lines present the solution obtained from the Haagerup-Larsen theorem, while the dashed line from the Abelization, where the only known analytical result corresponds to $r=1$. The agreement of the numerical results with the solution obtained from the Haagerup-Larsen theorem shows the validity of the cyclic approximation. The discrepancies at the edges are the effects of finite size of matrices.
\label{numerics1}}
\end{figure}

\subsection{Exact radial spectrum from the Haagerup-Larsen  theorem }  \label{sec:HaagLar}

We work in the cyclic approximation and $\bb{D}$ is a particular case of the circulant (permutation) matrix, its $T$ eigenvalues are just all complex roots of unity, i.e.  $\lambda_t =\exp(2\pi i \frac{t}{T})$. For large $T$ the eigenvalues tend to the uniform distribution on the unit circle, i.e. they approach the  constant measure  (Haar measure) of Circular Unitary Ensemble (CUE). Due to the Sylvester's determinant identity the matrices $\frac{1}{T}\bb{xDx}^{\dagger}$ and $\frac{1}{T}\bb{x}^{\dagger}\bb{xD}$ share the same non-zero eigenvalues, therefore the spectral problem of the time-lagged correlation matrix $\bb{C}$ is equivalent to the product of CUE ensemble and the anti-Wishart-type ensemble $\frac{1}{T} \bb{x}^{\dagger}\bb{x}= r \frac{1}{N} \bb{x}^{\dagger}\bb{x}$. The probability density function of the anti-Wishart ensemble is invariant under the unitary transformations, therefore its eigenbasis is already randomly oriented. The two matrices in the product are mutually free and we are allowed to use the Haagerup-Larsen theorem.

Since under the rescaling of an arbitrary matrix $\bb{X}$ by a real number $r$, $R_{r\bb{X}}(z)=rR_\bb{X}(rz)$, the R transform for  the matrix  $\bb{Y}={\frac{1}{T} \bb{x}^{\dagger}\bb{x}}$ reads $R_\bb{Y}(z)=rR_{\bb{aW}}(rz)= r \frac{r}{r-(zr)}= \frac{r}{1-z}$. We calculate its Green's function using \eqref{Rtransform}, which takes the form
\begin{equation}
G_{\bb{Y}}(z)=\frac{1-r+z+\sqrt{(1-r+z)^2-4z}}{2z}.
\end{equation}
The resolvent of a square of a matrix can be expressed by the Green's function of the matrix itself, with the help of the identity
\begin{equation}
G_{\bb{Y}^2}(z^2)=\frac{1}{2z}G_{\bb{Y}}(z)-\frac{1}{2z}G_{\bb{Y}}(-z),
\end{equation}
 which follows form the partial fraction decomposition. The S-transform, calculated via \eqref{Stransform} reads
 \begin{equation}
 S_{\bb{Y}^2}(z)=\frac{r+1+z}{(r+z)(r+2z+1)^2}.
 \end{equation}
 Using the Haagerup-Larsen theorem \eqref{eq:HaagLars}, we finally arrive at the cubic equation for the radial cumulative distribution function
 \begin{equation}
 4F^3+8F^2(r-1)+F(5(r-1)^2-s^2)+(r-1)^3-rs^2=0. \label{CDFequation}
 \end{equation}

 Upon choosing a real valued branch of the solution and taking into account the absence of zero modes in the original problem, one can calculate the spectral density
 \begin{equation}
 \rho(s)=\frac{1}{2\pi s r}\frac{dF}{ds}.
 \end{equation}
 As a shortcut, we also deduce from (\ref{CDFequation}) the values of the spectral radii (corresponding to $F=1$ and $F=0$)
\begin{equation}
s_{ext}=\sqrt{r(r+1)}, \qquad s_{int}=\frac{(r-1)^{3/2}}{\sqrt{r}}\theta(r-1), \label{eq:spectralradii}
\end{equation}
 where $\theta(x)$ is the Heaviside theta function. We remark that the same result can be obtained via calculation of the second and inverse second moments of the spectral density of anti-Wishart, as stated in the Haagerup-Larsen theorem. Several solutions compared to the numerical simulations are plotted in Fig. \ref{numerics1}. This result confirms the  diagrammatic calculation by~\cite{JAROSZLAG}.

\section{True-lagged correlation matrices and eigenvector correlators} \label{sec:diagrammatic}
\subsection{Non-hermitian random matrices}\label{sec:NonHermitianRandomMatrices}

All the above-mentioned approaches have  the same feature - they reduce the complex spectral problem to the calculation of real eigenvalues, either by imposing real spectra (symmetrization, Abelization) or by reducing the two-dimensional complex spectrum to a quasi-one dimensional case, imposing  azimuthal symmetry and considering only the radial variables (whitening, Haagerup-Larsen theorem). Moreover, securing the existence of the inverse of the delay matrix by adding additional non-zero elements (cyclic approximation) does not allow to probe the spectrum with varying lag depth, because the eigenvalues of such $\bb{D}$ remain the same, regardless of $\tau$.

In order to attack the truly non-Hermitian problem, we first have to find a representation of the complex Dirac delta, since the Sochocki-Plemelj formula ceases to work in the complex case. In the spirit of the electrostatic analogy, the applications of which to Random Matrix Theory were very fruitful, we use the Poisson kernel in two dimensions
\begin{equation}
\delta^{(2)}(z)=\lim_{\epsilon\to 0}\frac{1}{\pi}\frac{\epsilon^2}{(|z|^2+\epsilon^{2})^{2}}=\lim_{\epsilon\to 0} \frac{1}{\pi}\partial_{\bar{z}} \frac{\bar{z}}{|z|^2+\epsilon^2}.
\end{equation}
The differentiation with respect to $\bar{z}$ reduces the power of the denominator, which simplifies the object which one has to deal with. Let $\bb{X} $ be a random matrix, the mean spectral density of which we want to calculate. Denoting
\begin{equation}
g(z,\bar{z},w,\bar{w})=\< \frac{1}{N}\mbox{Tr} \frac{\bar{z} \One_{N} -\bb{X}^{\dagger}}{(z \One_N-\bb{X})(\bar{z} \One_{N}-\bb{X}^{\dagger})+|w|^2 \One_{N}}\>, \label{eq:gDef}
\end{equation}
the distribution of the eigenvalues on the complex plane can be calculated from~\cite{SOMMERS,FYODOROVSOMMERS,BROWN}
\begin{equation}
\rho(z,\bar{z})=\frac{1}{\pi}\lim_{|w|\to 0} \partial_{\bar{z}} g(z,\bar{z},w,\bar{w}).
\end{equation}

However, due to the nonlinearity in the denominator, it is very challenging to calculate $g$. It was realized in \cite{JANIKNOWAK,JARNOW}, that $g$ can be considered as a part of an extended object which is linear in $\bb{X}$ and therefore is easier accessible in practical calculations. Consider a $2N\times 2N$ matrix
\begin{equation}
\GG=\<\left(\begin{array}{cc}
z\One_{N} -\bb{X} & i\bar{w} \One_{N} \\
iw \One_{N} & \bar{z} \One_{N} -\bb{X}^{\dagger}
\end{array}\right)^{-1}\> = \< \left[ Q\otimes \One_{N} - {\cal X}\right]^{-1}\>,
\end{equation}
where $Q$ is a $2\times 2$ matrix representation of a quaternion and ${\cal X}$ is a duplicated matrix
\begin{equation}
Q=\left(\begin{array}{cc}
z & i\bar{w} \\
iw & \bar{z}
\end{array}\right), \qquad
{\cal X}=\left(\begin{array}{cc}
\bb{X} & 0 \\
0 & \bb{X}^{\dagger}
\end{array}\right). \label{eq:BlockStructure}
\end{equation}
We remark that in the literature on non-Hermitian random matrices there exists another representation of a quaternion where the off-diagonal elements are devoid of a factor of $i$ and the element above the diagonal appears with a minus sign. These representations are isomorphic, however throughout this paper we adapt the convention with imaginary units, which makes equations more symmetric.

The block structure reflects a compound nature of the considered object $\GG$. We therefore adopt a convention of writing its elements which is compatible with this structure and in terms of indices we write $\GG^{ij}_{\alpha\beta}$. The subscript Greek indices refer to the quaternionic nature, run from $1$ to $2$ and enumerate blocks of the duplicated matrix. Latin indices in the superscript enumerate elements of the matrix within each block. By taking the partial trace over the space of matrix indices (which is a block trace of the duplicated matrix) we produce a $2\times 2$ matrix which is itself a quaternion and we refer to this as the quaternionic Green's function (generalized Green's function)
\begin{equation}
G(Q)=\frac{1}{N}{\rm bTr} \GG=\left(\begin{array}{cc}
\<\frac{1}{N}\mbox{Tr}(\bar{z}\One_N-\bb{X}^{\dagger})\bb{D}^{-1}\> & \<\frac{-i \bar{w}}{N} \mbox{Tr} \bb{D}^{-1} \>\\
\<\frac{-i w}{N} \mbox{Tr} \bb{D}^{-1} \> & \<\frac{1}{N}\mbox{Tr} (z\One_N-\bb{X})\bb{D}^{-1} \>
\end{array}\right),
\end{equation}
where  $\bb{D}=(z\One_N-\bb{X})(\bar{z}\One_N-\bb{X}^{\dagger})+|w|^2\One_{N}$. In the analogy to the complex resolvent for Hermitian matrices, we give $\GG$ the name quaternionic resolvent. We identify the upper-left element of $G$ with \eqref{eq:gDef} and denote all its elements
\begin{equation}
G(Q)=\left(\begin{array}{cc}
g & i\bar{v} \\
i v & \bar{g}
\end{array}\right).
\end{equation}
The lower-diagonal element is just a complex conjugate copy of $g$, however, the off-diagonal elements in the $N\to \infty$ limit endow us for free with additional information.

If a non-Hermitian matrix is diagonalizable, it possesses a set of left $\bra{L_i}$ and right $\ket{R_i}$ eigenvectors, solving the eigenproblem
\begin{equation}
\bb{X}\ket{R_i}=\lambda_i \ket{R_i}, \quad \bra{L_i}\bb{X}=\bra{L_i} \lambda_i.
\end{equation}
They form a  biorthogonal set, i.e. they are normalized by the condition $\braket{L_i}{R_j}=\delta_{ij}$, however the left and right eigenvectors are not orthogonal among themselves, $\braket{L_i}{L_j}\neq \delta_{ij}\neq\braket{R_i}{R_j}$. The biorthogonality condition leaves a freedom of rescaling the eigenvectors by an arbitrary complex number $\ket{R_i}\to c_{i}\ket{R_i}$, $\bra{L_i}\to \bra{L_i}c_i^{-1}$, and also of multiplying by a unitary matrix $\ket{R_i}\to \bb{U}\ket{R_i}$, $\bra{L_i}\to\bra{L_i}\bb{U}^{\dagger}$. The simplest non-trivial quantity which is invariant under these transformations is the matrix of overlaps  $O_{\alpha\beta}=\braket{L_\alpha}{L_\beta}\braket{R_{\alpha}}{R_{\beta}}$, introduced by Chalker and Mehlig~\cite{CHALKERMEHLIG,CHALKERMEHLIG2}. The diagonal elements are the squares of the eigenvalue condition numbers, which play a significant role in the stability of the spectrum of non-normal matrices~\cite{WILKINSON,TREFETHEN,BERRY}. Chalker and Mehlig introduced a one point correlation function associated with the diagonal part of the overlap matrix
\begin{equation}
O_N(z)=\<\frac{1}{N^2} \sum_{i=1}^{N}O_{ii}\delta^{(2)}(z-\lambda_i)\>. \label{eq:Correlator}
\end{equation}
It naturally appears in non-Hermitian systems such as open chaotic scattering~\cite{SAVINSOK,FYODSAV,FYODOROVMEHLIG,BEENAKKER,BEENAKKER2} and also in diffusion on large non-Hermitian matrices~\cite{DIFFPRL,DIFFNPB}.

In the large $N$ limit the product of the off-diagonal elements of the quaternionic Green's function reproduces the eigenvector correlator \cite{HAAGLARSVECTORS,JANIKNOWAKCORR}
\begin{equation}
O(z)\equiv \lim_{N\to\infty} O_N(z)=\lim_{N\to\infty}\lim_{|w|\to 0} \frac{1}{\pi} |v|^2.
\end{equation}

A huge benefit of the linearization formalism comes from the fact that $\GG$ can be expanded into a geometric series in ${\cal Q}=Q\otimes \One_{N}$ and ${\cal X}$
\begin{equation}
\GG={\cal Q}^{-1}+\<{\cal Q}^{-1}{\cal X}{\cal Q}^{-1}\>+\<{\cal Q}^{-1}{\cal X}{\cal Q}^{-1}{\cal X}{\cal Q}^{-1}\>+\<{\cal Q}^{-1}{\cal X}{\cal Q}^{-1}{\cal X}{\cal Q}^{-1}{\cal X}{\cal Q}^{-1}\>+\ldots, \label{eq:MomentExpansion}
\end{equation}
provided that $||{\cal X}{\cal Q}^{-1}|| < ||\One_{2N}||$ in some norm. If we consider $z$ lying outside the spectrum of $\bb{X}$, then we are allowed to set $|w|$ to 0 and, as a consequence, $g$ is a holomorphic function. However, inside the spectrum we need to keep $w$ sufficiently large so that the expansion \eqref{eq:MomentExpansion} is legible. At the end of the calculations we set $|w|\to 0$ and then $g$ will depend both on $z$ and $\bar{z}$, indicating the non-vanishing spectral density.

The quaternionic Green's function, obtained as a block trace of \eqref{eq:MomentExpansion}, generates all mixed moments of non-Hermitian matrices in a similar manner as the complex Green's function for Hermitian matrices \eqref{eq:HermMoments}. Now, due to the fact that in general $\bb{X}$ does not commute with $\bb{X}^{\dagger}$, the class of possible moments is much broader. To show how they are encoded, we consider a quaternionic moment generating function, which we define as $M(Q)=Q^{-1}G(Q^{-1})Q^{-1}-Q^{-1}$, and its expansion into moments of $Q$
\begin{equation}
M(Q)=\<\frac{1}{N}{\rm bTr} {\cal X}\>+\<\frac{1}{N}{\rm bTr} {\cal X} {\cal Q} {\cal X}\>+\<\frac{1}{N}{\rm bTr} {\cal X}{\cal Q}{\cal X}{\cal Q}{\cal X} \>+\ldots \,\, . \label{eq:MMomentExP}
\end{equation}
$M$ is itself a $2\times 2$ matrix, the entries of which together with the entries of the quaternion we  denote here in a unifying way
\begin{equation}
M(Q)=\left(\begin{array}{cc}
M_{11} & M_{1\bar{1}} \\
M_{\bar{1}1} & M_{\bar{1}\bar{1}}
\end{array}\right),\quad
Q=\left(\begin{array}{cc}
Q_{11} & Q_{1\bar{1}} \\
Q_{\bar{1}1} & Q_{\bar{1}\bar{1}}
\end{array}\right).
\end{equation}

Suppose now that we would like to extract from $M(Q)$ the mixed moment $\<\frac{1}{N}\mbox{Tr} \bb{X}^{a_1}\bb{X}^{a_2}\ldots\bb{X}^{a_n}\>$, where each each of $a$'s is either $1$ or $\dagger$. The mnemotechnical rule, originating from the expansion \eqref{eq:MMomentExP}, goes as follows. First, one has to associate $\dagger$ with $\bar{1}$ and, obviously, $1$ with $1$. Then, identify first and last terms in the chain and consider the component $a_1a_n$  of $M$. In the next step one expands the appropriate component in powers of $z,\bar{z},iw,i\bar{w}$ and reads out the coefficient of $Q_{a_1a_2}Q_{a_2a_3}\ldots Q_{a_{n-1}a_n}$.

As an example we consider $\<\frac{1}{N}\mbox{Tr} \bb{XX}^{\dagger}\bb{X}^{\dagger}\bb{X}\>$. We have to expand $M_{11}$ into a series and consider the coefficient of $Q_{1\bar{1}}Q_{\bar{1}\bar{1}}Q_{\bar{1}1}=(i\bar{w}) \bar{z} (iw)$. The matrix $Q$ plays the role of a transition matrix in this chain and one can easily convince himself that all possible mixed moments are encoded in this way. We also remark that the quaternionic R-transform generates all possible mixed cumulants (a.k.a. connected moments) in the same way as $M$ generates all mixed moments~\cite{BURDASWIECH}. A particular type of mixed moments and its relation to the eigenvector correlation function $O(z)$ was studies thoroughly in~\cite{WALTERSSTARR}.

\subsection{Moment expansion of the quaternionic Green's function}

In this subsection we shall take the problem of determining the quaternionic Green's function of the lagged correlation matrix by expanding it into a power series and calculating the expectations of each terms. Then we find a way to sum all terms and write the Green's function in a neat form.

We remind that the lagged correlation matrix can be written as a matrix multiplication $\bb{C}^{\tau}=\frac{1}{T-\tau} \bb{X} \bb{D}\bb{X}^{\dagger}$. To find the spectrum of $\bb{C}^{\tau}$ and analyze its properties as the depth of the lag varies, we solve a more general problem. We consider a matrix $\bb{Y}=\frac{1}{T}\bb{XAX}^{\dagger}$, where $\bb{A}$ is an arbitrary $T\times T$ matrix independent of $\bb{X}$, with no symmetry constraints and not necessary invertible. The elements of $\bb{X}$ are complexes with both real and imaginary parts Gaussian random variables of $0$ mean and $1/2$ variance. The probability density function of $\bb{X}$ can be written in a concise form $P(\bb{X})\sim \exp(-\mbox{Tr} \bb{XX}^{\dagger})$. We are interested in the limit $T,N\to\infty$ with $r=N/T$ fixed, thus our results are valid also in the case when $\bb{X}$ is a real Gaussian matrix. However, for finite $N$ there is an accumulation of order of $N^{1/2}$ eigenvalues on the real line \cite{EDELMAN}.

We shall be performing calculation on $2N\times 2N$ matrices of the block structure \eqref{eq:BlockStructure}. The main object of interest is the quaternionic resolvent
\begin{equation}
\GG=\<[{\cal Q}-{\cal Y}]^{-1}\>,
\end{equation}
where ${\cal Q}=Q\otimes \One_{N}$ and
\begin{equation}
{\cal Y}=\left(\begin{array}{cc}
\frac{1}{T}\bb{XAX}^{\dagger} & 0 \\
0 & \frac{1}{T}\bb{XA}^{\dagger}\bb{X}^{\dagger}
\end{array} \right).
\end{equation}
There emerges a surprising symmetry stemming from the sandwich structure of $\bb{Y}$ which allows us to factorize ${\cal Y}=\frac{1}{T}{\cal X}{\cal A}{\cal X}^{\dagger}$, with
\begin{equation}
{\cal X}=\left(\begin{array}{cc}
\bb{X} & 0 \\
0 & \bb{X}
\end{array}\right), \qquad
{\cal A}=\left(\begin{array}{cc}
\bb{A} & 0 \\
0 & \bb{A}^{\dagger}
\end{array}\right), \qquad
{\cal X}^{\dagger}=\left(\begin{array}{cc}
\bb{X}^{\dagger} & 0 \\
0 & \bb{X}^{\dagger}
\end{array}\right).
\end{equation}

Since the real and imaginary parts of elements of $\bb{X}$ are centered Gaussian random variables, the expectation of higher moments can be calculated with the knowledge of the second cumulants only by means of Wick's (Isserlis') theorem. This property transfers simply to the language of the matrix elements and the second cumulants read
\begin{equation}
\<X_{it}X_{js}\>=0=\<X_{ti}^{\dagger}X_{sj}^{\dagger}\>, \qquad \<X_{it}X^{\dagger}_{sj}\>=\delta_{ij}\delta_{ts}.
\end{equation}
This brings an additional constraint in the moment expansion of $\GG$ that the only non-vanishing pairings are those where $\bb{X}$ is paired with $\bb{X}^{\dagger}$, thus the cumulants in the moment expansion are
\begin{equation}
\< {\cal X}_{\alpha\beta}^{at}{\cal X}^{\dagger sb}_{\mu \nu}\>=\delta_{\alpha\beta}\delta_{\mu \nu} \delta^{ab}\delta^{st}, \qquad \<{\cal X}_{\alpha\beta}^{at}{\cal X}_{\mu\nu}^{sb} \>=0 = \<{\cal X}^{\dagger at}_{\alpha\beta}{\cal X}_{\mu\nu}^{\dagger sb}\> . \label{eq:cumulants}
\end{equation}

The expansion of $\GG$ reads
\begin{equation}
\GG^{ab}_{\alpha\beta}=({\cal Q}^{-1})^{ab}_{\alpha\beta}+\<\frac{1}{T}({\cal Q}^{-1})^{ac}_{\alpha\gamma}{\cal X}^{cs}_{\gamma\delta}{\cal A}^{st}_{\delta\epsilon}{\cal X}^{\dagger td}_{\epsilon\varphi}({\cal Q}^{-1})^{db}_{\varphi\beta}\>+\ldots \,\, . \label{eq:Gexpansion}
\end{equation}
For the clarity of our formulas we adopt a convention that whenever two indices are repeated, we sum over the entire range of their variability.

Having \eqref{eq:cumulants}, we reduce any expectation involving strings of ${\cal X}$'s to the products of Kronecker deltas. Although the difficulty of calculating moments is avoided, the number of terms in the sum grows as $n!$, where $n$ is the order of expansion. Moreover, even for large $T$ some of the higher order terms give a contribution of order $\mathcal{O}(1)$, therefore one has to sum all terms carefully. In order to cope with the proliferation of indices, which in fact obscure the existing structure of the expansion \eqref{eq:Gexpansion}, we represent formulas as diagrams. The details of the diagrammatic expansion we present in Appendix \ref{sec:Diagrams}, where we thoroughly derive equations stemming from the pictorial structure. Here we briefly describe the procedure of calculations.

The moment expansion \eqref{eq:MomentExpansion} is a particular instance of the t'Hooft $1/N$ expansion in field theories with an internal $U(N)$ (or $O(N)$) group symmetry~\cite{THOOFT}. He showed that in the large $N$ limit only planar diagrams give the non-vanishing contribution, which simplifies the structure of diagrammatic expansion significantly.

Among planar diagrams produced by expressions in \eqref{eq:Gexpansion} we distinguish  a class of one-line irreducible (1LI, in physics literature they are also known as one-particle irreducible) diagrams. We denote $\Sigma^{ab}_{\alpha\beta}$ a sum over all possible 1LI planar diagrams with amputated external legs and refer to it as self-energy. These diagrams are the building block of $\GG$, which can be expressed as a series, the $n-$th term of which contains $n$ 1LI diagrams. This relation is known as the Schwinger-Dyson equation and reads
\begin{equation}
\left({\cal Q}_{\alpha\gamma}^{ac}-\Sigma^{ac}_{\alpha\gamma}\right)\GG^{cb}_{\gamma\beta}=\delta^{ab}\delta_{\alpha\beta}. \label{eq:SD1}
\end{equation}

Following the ideas from \cite{BURDAJURKIEWICZ}, we introduce a dual $2T\times 2T$ matrix, which differs from ${\cal Y}$ by a cyclic permutation of factors. It is defined as $\hat{{\cal Y}}=\frac{1}{T}{\cal X}^{\dagger}{\cal X}{\cal A}$. We distinguish all matrices and functions corresponding to the dual problem by putting a hat over them. We remark that contrary to the instance in \cite{BURDAJURKIEWICZ}, the dual matrix is only an auxiliary object and it does not correspond to any non-Hermitian matrix problem because the diagonal blocks are not Hermitian conjugates of each other.

We consider a moment expansion of the dual problem
\begin{equation}
\hat{\GG}=\<\left[ Q\otimes \One_{T}-\hat{{\cal Y}}\right]^{-1}\>,
\end{equation}
for which we can write the Schwinger-Dyson equation as in the previous case
\begin{equation}
\left(Q_{\alpha\gamma}\delta^{tr}-\hat{\Sigma}^{tr}_{\alpha\gamma}\right)\hat{\GG}^{rs}_{\gamma\beta}=\delta^{ts}\delta_{\alpha\beta}. \label{SD2}
\end{equation}

With the help of the dual problem we can relate $\GG$ and $\hat{\GG}$ with self-energies, obtaining a closed system of equations. An analysis of the corresponding diagrams yields
\begin{eqnarray}
\Sigma_{\alpha\beta}^{ab}&=&\frac{1}{T}\delta^{ab}{\cal A}_{\alpha\gamma}^{cd}\hat{\GG}_{\gamma\mu}^{dc} Q_{\mu\beta}, \\
\hat{\Sigma}_{\alpha\beta}^{ts}&=&\frac{1}{T}Q_{\alpha\gamma}\GG^{cc}_{\gamma\mu}{\cal A}^{ts}_{\mu\beta}.
\end{eqnarray}

Substitution of self-energies to the Schwinger-Dyson equations gives us two coupled matrix equations

\begin{eqnarray}
\left(Q_{\alpha \nu}\delta^{ac}-\frac{1}{T}\delta^{ac}{\cal A}^{ts}_{\alpha\gamma}\hat{\GG}_{\gamma\mu}^{st}Q_{\mu\nu}\right)\GG_{\nu\beta}^{cb}=\delta_{\alpha\beta}\delta^{ab},  \label{eq:MatrixCoupledFirst} \\
\left(Q_{\alpha\nu}\delta^{tr}-\frac{1}{T}Q_{\alpha\gamma}\GG^{cc}_{\gamma\mu}{\cal A}^{tr}_{\mu\nu}\right)\hat{\GG}^{rs}_{\nu\beta}=\delta_{\alpha\beta}\delta^{ts}.
\end{eqnarray}

Keeping in mind that $G_{\alpha\beta}=\frac{1}{N}\GG^{aa}_{\alpha\beta}$ is the quaternionic Green's function, equation \eqref{eq:MatrixCoupledFirst} imples that $\GG=G\otimes \One_{N}$. Eliminating $\hat{\GG}$ from the equations above, we obtain the final equation for the quaternionic Green's function, written in the matrix form
\begin{equation}
\left[Q-\frac{1}{T}{\rm bTr}\left({\cal A}[\One_{2T}-r(G\otimes\One_{T}){\cal A}]^{-1}\right)\right]G=\One_{2}. \label{eq:Solution}
\end{equation}
 This equation is exact in the limit $N,T\to\infty$ with $N/T=r$ constant, although for finite size of matrices there are $1/N$ corrections. Simple algebraic manipulations  bring this equation to the following form
\begin{equation}
[Q-M_{\bb{A}}(rG)]G=\One_{2},
\end{equation}
where $M_\bb{A}(Q)$ is the quaternionic moment generating function of $\bb{A}$ \eqref{eq:MMomentExP}, showing that the spectral properties of $\frac{1}{T}\bb{XAX}^{\dagger}$ are completely determined by the mixed moments of $\bb{A}$.
 If additionally the matrix $\bb{A}$ is invertible, defining $G_{\bb{A}^{-1}}=\frac{1}{T}{\rm bTr} (Q\otimes \One_T-{\cal A}^{-1})^{-1}$, the quaternionic Green's function of $\bb{A}^{-1}$, the above equation can be written in another neat form
 \begin{equation}
 \left[Q+G_{\bb{A}^{-1}}(rG)\right]G=\One_{2}. \label{eq:SolutionSimplified}
 \end{equation}

\section{Analysis of the spectrum of the lagged correlation matrix} \label{sec:Analysis}
\subsection{Unit time lag}

We consider a unit time shift $\tau=1$ and the standard limits $N,T\to\infty$ with $r=N/T$ fixed. As in Section \ref{sec:HaagLar} we replace the unit shift matrix with a circulant matrix, the spectral density of which in the limit $N\to \infty$ covers uniformly the unit circle and the matrix itself is normal, therefore its Green's function is the same as for the Haar unitary $\bb{U}$. This object was calculated in \cite{JAROSZGOERLICH,JAROSZUNIT} and reads
\begin{equation}
G_{\bb{U}}\left(\begin{array}{cc}
z & i\bar{w} \\
iw & \bar{z}
\end{array}\right)=\left(\begin{array}{cc}
\frac{1}{2z}\left(1+\frac{|z|^2-|w|^2-1}{\sqrt{\alpha^2-4|z|^2}}\right) & -\frac{iw}{\sqrt{\alpha^2-4|z|^2}} \\
-\frac{i\bar{w}}{\sqrt{\alpha^2-4|z|^2}} & \frac{1}{2\bar{z}}\left(1+\frac{|z|^2-|w|^2-1}{\sqrt{\alpha^2-4|z|^2}}\right)
\end{array}\right),
\end{equation}
where $\alpha=|z|^2+|w|^2+1$. Endowed with this knowledge we can use the functional matrix equation \eqref{eq:SolutionSimplified}, obtaining a system of complex algebraic equations
\begin{eqnarray}
1&=&gz-v\bar{w}+\frac{1}{2r}\left(1+\frac{r^2(|g|^2-|v|^2)-1}{\sqrt{\beta^2-4|g|^2r^2}}\right)+\frac{rv^2}{\sqrt{\beta^2-4|g|^2r^2}},\label{eq:unit1}\\
0&=&z\bar{v}+\bar{w} g+\frac{\bar{v}}{2rg}\left(1+\frac{r^2(|g|^2-|v|^2)-1}{\sqrt{\beta^2-4|g|^2r^2}}\right)-\frac{r\bar{g} v}{\sqrt{\beta^2-4|g|^2 r^2}},\label{eq:unit2}
\end{eqnarray}
with $\beta=r^2(|g|^2+|v|^2)+1$. The additional two equations from the $2\times 2$ matrix are the complex conjugate of the ones above. Nevertheless, in calculations it is convenient to treat $g$ and $\bar{g}$ as independent variables and solve the extended system of 4 equations. The two additional equations assert that the solutions for $g$ and $\bar{g}$ are mutually conjugated.

Having performed calculations which involve expansion into moments, one can put $w\to 0$ in further computations. It is clear that $v=0$ is a solution and  equation \eqref{eq:unit1} reduces to
\begin{equation}
1=gz+\frac{1}{2r}\left(1+\frac{r^2|g|^2-1}{\sqrt{(r^2|g|^2-1)^2}}\right).
\end{equation}
The square root yields an absolute value, which produces two holomorphic solutions
\begin{eqnarray}
g=\frac{1-r}{rz} & \mbox{for} & |z|\leq r-1, \\
g=\frac{1}{z} & \mbox{for} & |z| > r-1.
\end{eqnarray}

Bounds for the validity of these two solutions do not determine where the spectral density vanishes. In order to find the borderline of the spectrum, we need a non-holomorphic solution. To this end, we multiply \eqref{eq:unit2} by $g/\bar{v}$ and subtract it from \eqref{eq:unit1}, obtaining
\begin{equation}
\frac{\bar{v}}{v}=r\frac{|v|^2+|g|^2}{\sqrt{\beta^2-4|g|^2r^2}}.
\end{equation}
Since the r.h.s. is real, $v$ is either real or purely imaginary. The choice of the solution is equivalent to the choice of the $2\times 2$ matrix representation of a quaternion \eqref{eq:BlockStructure}. In our convention we take $v$ real and get the equation for $g$
\begin{equation}
4g^3r^3z^3+4g^2z^2r^2(1-r)+gzr\left((1-r)^2-|z|^2\right)-|z|^2=0.
\end{equation}
The structure of the equation above suggests to substitute $f=gz$, which brings it to the following form
\begin{equation}
4f^3r^3+4f^2r^2(1-r)+fr\left((1-r)^2-|z|^2\right)-|z|^2=0. \label{eq:unitCDF}
\end{equation}
It is clear that $f$ depends only on $|z|$, which means that this is a radial cumulative distribution function for matrix ensembles, the spectrum of which is rotationally invariant on the complex plane \cite{FEINBERGZEE}. The spectral density is then given by
\begin{equation}
\rho(z,\bar{z})=\frac{1}{\pi}\partial_{\bar{z}}g(z,\bar{z})=\frac{1}{2\pi |z|}f'(|z|).
\end{equation}
Moreover, the eigenvector correlator \eqref{eq:Correlator}, calculated from the system of equations \eqref{eq:unit1} \eqref{eq:unit2} inside the spectrum reads
\begin{equation}
O(|z|)=\frac{1}{\pi}\frac{1}{2fr^2+r-r^2}-\frac{f^2}{\pi |z|^2},
\end{equation}
 outside the spectrum the correlator vanishes.

It is worth reminding that the obtained spectrum of a sandwiched matrix differs from the one of a product of the anti-Wishart matrix and a Haar unitary by an absence of $T-N$ zero modes. Indeed, a transformation $F=(1-r)+fr$ brings \eqref{eq:unitCDF} to \eqref{CDFequation}. These two matrices, obtained by a product of 3 matrices but in a different order, share the same spectral properties, which is a consequence of the Sylvester's determinant identity. However, the block trace operation is not cyclic, therefore the eigenvector correlator distinguishes between these ensembles. This correlator for the product of anti-Wishart with Haar unitary, which falls into the range of applicability of the Haagerup-Larsen theorem, can be calculated from a simple formula \eqref{eq:HaagLarsCorr}.

To calculate the  boundary of the spectrum we exploit the fact that $f$ is a radial cumulative distribution function and $f=1$ corresponds to the outer radius. If $N<T$ there are no zero modes, and the inner spectral radius is determined by the condition $f=0$, while for $N>T$ the presence of zero eigenvalues requires that $f=1-1/r$ gives the inner radius. Their values agree exactly with \eqref{eq:spectralradii}, thus we can finally write the spectral density
\begin{equation}
\rho(z,\bar{z})=\max\left(\frac{r-1}{r},0\right) \delta^{(2)}(z)+\frac{1}{2\pi |z|}f'(|z|)\theta(|z|-s_{int})\theta(s_{ext}-|z|),
\end{equation}
where $f$ is a real valued branch of the solution of \eqref{eq:unitCDF} and $s_{int}, s_{ext}$ are given by \eqref{eq:spectralradii}.

\subsection{Going deeper with the lag}

Considering a deeper lag, one introduces an additional time scale to the problem, namely, the depth of the lag $\tau$. In the standard limiting procedure $N,T\to\infty$, keeping $\tau$ constant does not produce new results. The spectral density and the eigenvector correlator tend to the already solved case of the unit time shift. This model works well if the time lag is much smaller than the length of time series.

In real situations time lags sometimes can be comparable with the temporal length of the data. In order to capture this property, we perform  the limiting procedure when all $N,T,\tau$ tend to infinity with $r=N/T$ and additionally $\beta=\tau/T$ fixed. For the convenience we introduce another parameter $\alpha=(1-\tau/T)^{-1}$ which is finite in this limit.

The algebraic equation for the generalized Green's function of the lagged correlation matrix in this double scaling limit can be easily obtained from \eqref{eq:Solution}, by taking into account that $\frac{1}{T-\tau}\bb{D}=\frac{1}{T}\alpha \bb{D}$ and it takes the following form
\begin{equation}
\left[Q-\frac{\alpha}{T}{\rm bTr}\left({\cal D}[\One_{2T}-\alpha r(G\otimes\One_{T}){\cal D}]^{-1}\right)\right]G=\One_{2}, \label{eq:algebraicLAg}
\end{equation}
with $D_{ts}=\delta_{t+\tau,s}$ and
\begin{equation}
{\cal D}=\left(\begin{array}{cc}
\bb{D} & 0 \\
0 & \bb{D}^{T}
\end{array}\right).
\end{equation}

This equation simplifies considerably if $\tau/T$ is a fraction with a small denominator. The delay matrix can then be written as a Kronecker product of a low dimensional matrix and the identity matrix. Here, we present a step-by-step derivation for the simplest instance $T=2\tau$. It is convenient to write matrices as follows
\begin{equation}
{\cal D}=\left(\begin{array}{cccc}
0 & 1 & 0 & 0 \\
0 & 0 & 0 & 0 \\
0 & 0 & 0 & 0 \\
0 & 0 & 1 & 0
\end{array}\right)\otimes \One_{\tau},\, G\otimes\One_{T}=\left(\begin{array}{cccc}
g & 0 & i\bar{v} & 0 \\
0 & g & 0 & i\bar{v} \\
iv & 0 & \bar{g} & 0 \\
0 & iv & 0 & \bar{g}
\end{array}\right)\otimes \One_{\tau}, \, \One_{2T}=\One_{4}\otimes\One_{\tau},
\end{equation}
which reduces the matrix within the block trace in \eqref{eq:algebraicLAg} to a Kronecker product of a $4\times 4$ matrix with the identity matrix, simplifying considerably the matrix inversion and the block trace operation. Upon setting $w=0$, one obtains a system of complex equations
\begin{equation}
\frac{2r|v|^2}{1+4r^2|v|^2}+gz=1,\qquad \bar{v}\left(\frac{2r\bar{g}}{1+4r^2|v|^2}-z\right)=0.
\end{equation}
Once again $v=0$ together with $g=1/z$ is a solution valid for large $|z|$. Assuming $v\neq 0$, we eliminate $|v|^2$, obtaining the quadratic equation for $f=gz$
\begin{equation}
4f^2r^2+2fr(1-2r)-|z|^2=0.
\end{equation}
Substitution $f=1$ yields the spectral radius $s_{ext}=\sqrt{2r}$. Taking appropriate branch of solutions and differentiating with respect to $|z|$, one obtains the spectral density
\begin{equation}
\rho(z,\bar{z})=\frac{1}{2\pi r \sqrt{(1-2r)^2+4|z|^2}} \theta(\sqrt{2r}-|z|).
\end{equation}
The eigenvector correlator in this case is given by the formula
\begin{equation}
O(z,\bar{z})=\frac{2r-1-2|z|^2+\sqrt{(2r-1)^2+4|z|^2}}{8\pi r^2|z|^2}\theta(\sqrt{2r}-|z|).
\end{equation}

The spectral density is the same as the one for a product of two independent rectangular Wishart matrices \cite{BJLNSproducts}. We argue here that this is the case for all $\tau\geq T/2$. Let us write matrices in the block form
\begin{eqnarray}
\bb{X}=\left(\bb{A}_{N\times\tau},\bb{B}_{N\times(T-\tau)}\right),\quad \bb{D}(\tau)=\left(\begin{array}{cc}
0_{\tau\times(T-\tau)} & \One_{\tau\times\tau} \\
0_{(T-\tau)\times(T-\tau)} & 0_{(T-\tau)\times\tau}
\end{array}\right), \\
 \bb{X}^{\dagger}=\left( \begin{array}{c}
\bb{F}_{(T-\tau)\times N} \\
\bb{H}_{\tau\times N}
\end{array}\right).
\end{eqnarray}

In this representation the lagged correlation matrix takes the form $\bb{C}=\frac{1}{T-\tau}\bb{AH}$. For $\tau\geq T/2$ blocks $\bb{A}$ and $\bb{H}$ have no common elements, and the lagged correlation matrix reduces to the product of two independent  rectangular matrices of size $N\times \tau$ multiplied by a factor $(T-\tau)^{-1}$. A RMT benchmark for the lagged correlation matrix as a product of two independent rectangular Gaussian matrices, as proposed by Livan \cite{LIVAN}, turns out to be valid for $\tau\geq T/2$.

Considering various lags such that the denominator of $\tau/T$  is greater or equal to 3, one obtains a system of coupled polynomial equations for $f=gz$ and $|v|^2$, which cannot be solved analytically. Nevertheless, the numerical solutions agree with Monte Carlo simulations, as presented in Fig. \ref{fig:VariousLags}.

\begin{figure}
\includegraphics[width=0.49\textwidth]{fig_density_various_lags.pdf} \includegraphics[width=0.49\textwidth]{fig_corr_various_lags}
\caption{Radial spectral densities (a) and the eigenvector correlator (b) obtained by the diagonalization of $4000$ Gaussian lagged correlation matrices of size $N=1000$ with various rectagularities and lag depths (dots) juxtapposed with the numerical solutions of the algebraic equation \eqref{eq:algebraicLAg}. The discrepancies near the edges of the spectrum are the effects of the finite size of matrices.
\label{fig:VariousLags}}
\end{figure}

Even more important from the perspective of data analysts is the radius of a circle bounding the support of the spectral density. By imposing matching conditions of the holomorphic and non-holomorphic solutions for $g$ we obtain an implicit equation for the external radius $s_{ext}$
\begin{equation}
\sum_{k=1}^{M-1}\left(\frac{\alpha r}{s_{ext}}\right)^{2k}(1-k\beta)=r, \label{eq:SpecRad}
\end{equation}
where $\beta=\tau/T$, $M=\left\lceil \frac{T}{\tau} \right\rceil$ and $\left\lceil x\right\rceil$ denoted the ceiling function. Details of the derivation are relegated to Appendix \ref{sec:SpectralRadius}, plots depicting the solutions are presented in Fig. \ref{fig:SpecRad}. In the limit $\tau/T\to 0$ equation \eqref{eq:SpecRad} yields $s_{ext}=\sqrt{r(r+1)}$, reproducing the result for the unit time lag.

\begin{figure}
\begin{center}
\includegraphics[width=0.7\textwidth]{fig_radius_vs_lag.pdf} \end{center}
\caption{Dependence of the spectral radius as a function of the lag depth for 3 arbitrary rectangularities. For $\tau>T/2$ there is a universal behavior  $s_{ext}=\sqrt{\alpha r}$. The cusps are consequences of the discontinuity of the ceiling function.
\label{fig:SpecRad}}
\end{figure}

\section{Conclusions } \label{sec:Conclusions}

In this paper we have analyzed a broad class of large lagged correlation matrices, using the tools of random matrix theory.
We started from the comparison of several  previously studied cases (symmetrization, Abelization, whitening)  using the powerful tools of free random variables. This approach has not only allowed us to rederive  the already existing results, but also to resolve some controversies in the literature. Then, we considered the non-Hermitian problem of the lagged correlation matrix using the concise description of the diagrammatic techniques in terms of the so-called generalized Green's functions (quaternionic Green's function).  Two  new results include: (i) the algebraic equation for the quaternionic resolvent \eqref{eq:Solution} and (ii) the solution  for deeper time lags - in particular the implicit expression for the external radius of the complex spectrum \eqref{eq:SpecRad}. The solution for the lagged correlation matrix is obtained as a particular instance of a more general random matrix problem, namely the non-Hermitian generalization of the Wishart ensemble.
Additionally, we have addressed the issue of the eigenvector correlators in such matrices. To the best of our knowledge, such objects were not studied so far in non-Hermitian generalizations of multivariate statistics, perhaps due to the lack of the pertinent mathematical methods. We hope that our framework will fill this gap.

 Last but no least, several generalizations of the presented formalism  are possible.  The  first is the inclusion of spatio-temporal correlations for time-lagged covariances, which will cause the spectra to lose the rotational symmetry. An important step in this direction was already done~\cite{JAROSZLAG}, but resulting formalism is so far quite complicated. The other  direction  is the departure from Gaussianity in favor of heavy tailed data, so abundant in real-world complex systems.

The intention of the paper was also to be self-consistent and  to promote new techniques for large lagged matrices, therefore we decided to present the detailed calculations and to add extensive appendices.  We also successfully confronted  formulae obtained with numerical, synthetic data.
The presented formalism can be applied  to the empirical analysis in any field where lagged correlations are essential - we have already outlined the domains of science in the introduction. From our point of view, the most interesting are the  hidden correlations in  biomedical data and we will address this issue in the sequel to this work~\cite{CHIALVO}.

\subsubsection*{Acknowledgments}
We thank  Zdzis\l aw Burda, Romuald Janik  and Andrzej Jarosz for discussions and for the check of some of the calculations at the early stage of this work. This work was supported  by the Grant DEC-2011/02/A/ST1/00119 of the National Center of Science.
WT appreciates also the support from Polish Ministry of Science and Higher Education through the Diamond Grant 0225/DIA/2015/44 and the scholarship of Marian Smoluchowski Research Consortium Matter Energy Future from
KNOW funding.