Polyostotic sclerosing histiocytosis, also known as Erdheim-Chester disease (ECD), is a rare form of non-Langerhans histiocytosis. ECD has wide clinical spectrums which mainly affect skeletal, neurological, dermatological, retroperitoneal, cardiac, and pulmonary manifestations. Here we describe a case of ECD in a 45-year-old female who presented initially with bilateral knee pain and homonymous superior quadrantanopia progressed to ophthalmoplegia and complete visual loss of the left eye over a period of one year. Plain X-ray of both knees showed bilateral patchy sclerosis of the distal femur and upper parts of the tibiae. Initial brain magnetic resonance imaging (MRI) showed bilateral enhancing masses in the temporal lobes anterior to the temporal horns, thickening of the pituitary stalk, partially empty sella, and involvement of the left cavernous sinus one year later. Our case is a peculiar case of ECD initially presented with unilateral homonymous superior quadrantanopia due to involvement of the visual apparatus in the mesial temporal lobe which progressed to unilateral ophthalmoplegia and total visual loss secondary to involvement of the cavernous sinus. Thus, the diagnosis of ECD should be kept in mind in the presence of bilateral bone sclerotic lesions.