Why doesn't the cytosol dissolve the polar structures?
Many cell components are not simply hydrophobic or hydrophilic, but have dual affinities. Proteins typically have structures which result in the interior of the protein being hydrophobic and the exterior, which is exposed to the water in the cytosol, being hydrophilic. Thus, differences in polarity between different regions allow proteins to be dissolved in the cytosol while still maintaining a stable structure.

Similarly, the lipid bilayer is composed of phospholipids, which can be described as "tails" of hydrophobic carbon chains connected to hydrophilic phosphate "heads". The phospholipid bilayer is organized so that the phosphate heads are on the surface of each side of the bilayer, with the carbon chains from each side of the bilayer intertwined with each other in the interior.