What type of correlation is this equation? In a book [1], gives an equation (equation 5.31 in the book) for estimating frequency offset in OFDM systems proposed by Classen. the procedure is: 


  first, two OFDM symbols, $y_l [n]$ and $y_{l+D} [n]$ , are saved in a memory after synchronization. then, the signals are transformed into ${Y_l [k]}_{k=0}^{N-1}$ and ${Y_{l+D} [k]}_{k=0}^{N-1}$ via FFT, from which pilot tones are extracted. After estimating CFO from pilot tones in the frequency domain, the signal is compensated in the time domain. 


Here is the equation: 

$ \hat \epsilon_{acq} = \dfrac{1}{2\pi \cdot T_{sub}} \underset{\epsilon}{max} \left \{ \left \vert \sum_{j=0}^{L-1} Y_{l+D}[p[j],\epsilon]\cdot Y_{l}^{*}[p[j],\epsilon]\cdot X_{l+D}^{*}[p[j]]\cdot X_l[p[j]] \right \vert \right \}  $      

where $L$, $p[j]$, and $X_{l}[p[j]]$ are the number of pilot tones, the location of the jth pilot tone, and the pilot tone located at $p[j]$ in the frequency domain at the $l^{th}$ symbol period, respectively.

From the equation, It's supposed to find the normalized CFO, $\epsilon$. but, the parameter we are finding is involved in the equation $[P[j],ε]$.
My question is: how this parameter we're supposed to estimate is in the middle of the equation? What type of correlation is this? Or did I misunderstand the equation? Please explain.    


[1] Yong Soo Cho et al., "MIMO-OFDM wireless communications with MATLAB", John Wiley &amp; Sons (asia), 2010
I understand your confusion because the equation is barely understandable from the information given in the book. It becomes more clear from the original paper by Classen and Meyr [1] from which it has been taken. They propose a two stage frequency offset estimation that consists of an acquistion stage and a tracking stage. The equation you've cited represents the acquisition algorithm. In fact, it is a searching algorithm that "tries out" various values of frequency offset $\epsilon$ and chooses the one value $\hat\epsilon_\mathrm{acq}$ that yields the maximum correlation sum. It can be described as follows



For every $\epsilon$


Compensate the received time domain signal for the supposed frequency offset $\epsilon$
Calculate the DFT of the frequency offset compensated RX signal. This yields $Y_{l}[p[j],\epsilon]$
Calculate the correlation sum $R_\epsilon$. (The term inside $|\cdots|$)


Then, find the maximum $\hat R_\epsilon$ of all $R_\epsilon$ and calculate the estimated frequency offset by 
$$
\hat \epsilon_{acq} = \dfrac{\hat R_\epsilon}{2\pi \cdot T_{sub}}
$$



Of course it's not feasible to evaluate the correlation sum for every $\epsilon$ because the number of possbile values is infinite. Instead, you have to restrict the range of values to a reasonable frequency offset that is determined by the hardware you're using. Additionaly, this "search range" has to be partitioned into discrete values. The authors of [1] note


  In practice we found that it is sufficient to space the trial parameters $0.1/T_{sub}$ apart from each other.


Where $T_\mathrm{sub}$ is the length of one OFDM symbol including guard interval.



[1] Classen, F. and Myer, H. (June 1994) Frequency synchronization algorithm for OFDM systems suitable for
communication over frequency selective fading channels. IEEE VTC’94, pp. 1655 1659.