Let $G$ be a group and $H \leq G$. Suppose $gH = Hg, \forall g \in G$. We know that this is equivalent to $gHg^{-1} \subseteq H, \forall g \in G$ and $gHg^{-1} = H, \forall g \in G$. However I haven't seen any textbook write what was stated in the title. Is it false?