Why some parts of the human body have immune privilege?
The immune privilege is happening for brain, CNS, eye, testis and the uterus. All these tissues (or structures) share that they cannot be regenerated once they get damaged by an overshooting immune reaction. Infections in the eye followed by a strong immune response often lead to the loss of the eye, in rodents loss if immune tolerance against the fetus leads to abortion. 

So basically this is a protective mode for these tissues to protect them from a stong immune reaction. See the references for some more details (especially on how this works).

References:


See no evil, hear no evil, do no evil: the  lessons of immune
privilege
What is immune privilege (not)?
Immune privilege or privileged immunity?
Immune Privilege and the Philosophy of Immunology


  Each immune privileged tissue has a unique function. The eye must protect the light path and signals that stimulate the retina, and photoreceptors to preserve vision. The testis has to protect the sperm as they proceed to the epididymis where they mature. The maternal reproductive tract has to protect its eggs both before and after fertilization and thus has developed specialized mechanisms to modify the body’s response to foreign antigens. These unique challenges require different solutions and lead to unique immune privilege mechanisms. However, although microenvironment and the stromal cells that carry out the particular function may vary between tissues, and consequently the mechanisms that promote regulation may be unique to that cell or tissue, the goal is the same: limit collateral damage to preserve tissue integrity and maintain homeostasis to the extent possible, without compromising host defense.