Channel estimation using bit reversed order of FFT matrix
So, according to your text,
$$ H := F^bh,$$
i.e. what you call the bit-reversed order FFT of the channel impulse response.
Since &quot;bit-reversed order&quot; just implies you're permuting outputs of the FFT, $F^b$ is just a row permutation $\Pi_{BR}$ of the standard DFT matrix $D$.
Since the &quot;proper&quot; DFT actually leads to a diagonal $H$ (as a linear channel never &quot;mixes&quot; any other frequencies in), you already know exactly what to do to convert your $H$ to a diagonal matrix:
Reverse the reverse bit ordering:
$$\Pi^{-1}_{BR}H=\Pi^{-1}_{BR}\left(F^bh\right) = \Pi^{-1}_{BR}\left(\Pi_{BR}Dh\right) = Dh,$$
which, in best classical OFDM nature, diagonalizes the channel.
(Using the full rank of any permutation matrix, you can also show this is the only way to diagonalize the result of the bit-reverse ordered FFT; putting this into a ray of optimism: Those who set out to invent OFDM will in expectation invent OFDM!.)
Channel estimation relates to the way you want to correct the channel influence. Hence, once you diagonalized your system, it becomes &quot;normal&quot; OFDM, and all the normal OFDM methods applies.