An open question in cell biology is how the general intracellular transport machinery is adapted to perform specialized functions in polarized cells such as neurons. Here we illustrate this adaptation by elucidating a role for the ubiquitous small GTPase Ras-related protein in brain 5 (Rab5) in neuronal polarity. We show that inactivation or depletion of Rab5 in rat hippocampal neurons abrogates the somatodendritic polarity of the transferrin receptor and several glutamate receptor types, resulting in their appearance in the axon. This loss of polarity is not caused primarily by increased transport from the soma to the axon but rather by decreased retrieval from the axon to the soma. Retrieval is also dependent on the Rab5 effector Fused Toes (FTS)-Hook-FTS and Hook-interacting protein (FHIP) (FHF) complex, which interacts with the minus-end-directed microtubule motor dynein and its activator dynactin to drive a population of axonal retrograde carriers containing somatodendritic proteins toward the soma. These findings emphasize the importance of both biosynthetic sorting and axonal retrieval for the polarized distribution of somatodendritic receptors at steady state.