Congenital penile anomalies (CPAs) are among the most common human birth defects. Reports of CPAs, which include hypospadias, chordee, micropenis, and ambiguous genitalia, have risen sharply in recent decades, but the causes of these malformations are rarely identified. Both genetic anomalies and environmental factors, such as antiandrogenic and estrogenic endocrine disrupting chemicals (EDCs), are suspected to cause CPAs; however, little is known about the temporal window(s) of sensitivity to EDCs, or the tissue-specific roles and downstream targets of the androgen receptor (AR) in external genitalia. Here, we show that the full spectrum of CPAs can be produced by disrupting AR at different developmental stages and in specific cell types in the mouse genital tubercle. Inactivation of AR during a narrow window of prenatal development results in hypospadias and chordee, whereas earlier disruptions cause ambiguous genitalia and later disruptions cause micropenis. The neonatal phase of penile development is controlled by the balance of AR to estrogen receptor a (ER alpha) activity; either inhibition of androgen or augmentation of estrogen signaling can induce micropenis. AR and ER alpha have opposite effects on cell division, apoptosis, and regulation of Hedgehog, fibroblast growth factor, bone morphogenetic protein, and Wnt signaling in the genital tubercle. We identify Indian hedgehog (Ihh) as a novel downstream target of AR in external genitalia and show that conditional deletion of Ihh inhibits penile masculinization. These studies reveal previously unidentified cellular and molecular mechanisms by which antiandrogenic and estrogenic signals induce penile malformations and demonstrate that the timing of endocrine disruption can determine the type of CPA.