How does a person who wears prism glasses through which the world is perceived upside-down experience the transition back to normal? If someone wears prism glasses that turn the visual field upside down (see this video), this person will eventually adapt and see the world in its normal orientation again. Now my question is, how does the person wearing these glasses perceive the transition from upside-down back to normal after the prism glasses are taken off?
Short answer
Based on the papers I've found I conclude that visual perceptions are not grossly altered after wearing prism glasses for extended periods. Instead, adaptation to prism glasses is mainly brought about by the adjustment of motor behavior to compensate for the shifted visual input. 

Background
In a review on the topic of adaptation effects in response to wearing prism glasses, Harris concludes that it is the 'position sense' that adapts, and not the visual system (Harris, 1965). In the more recent literature the adaptation to the shifts, or reversals in vision brought about by prism glasses is indeed ascribed to the adaptation of the proprioceptive system, i.e., subjects adapt their motor behavior to compensate for the visual field shift (e.g., Richter et al. (2002), Hatada &amp; Rosetti (2006)), or through shifts in their egocentric reference frame although subtle, short-lasting after-effects do seem to occur in the visual domain too (Hatada et al., 2006). 
Note that the experiments described above, human subjects were used and due to practical and ethical reasons, subjects did not wear the prism glasses longer than a few hours up to 1 week at max.

In animal studies, much longer prism-wearing times have been applied. In a study where monkeys wore left-right shifting prism goggles for 90 days, the ipsilateral visual cortex was found to be activated (Sugita et al., 1996). Normally, only the contralateral cortex is activated by ipsilateral stimuli.  Hence, here in these experiments a definite neurophysiological correlate of visual adaptation was found. In four human subjects that wore similar goggles for about 36 days it was subsequently shown that also their ipsilateral cortices were activated, but only after 1 to 2 weeks (Miyauchi et al., 2004). However, the authors of this study don't report the after-effects, but they do conclude that:


  [The] changes observed [in] adaptation to left–right reversing may be associated with the process of [...] strategic perceptual-motor control. One of the most crucial actions for a subject with left–right reversed vision is controlling the hands [...]. For example, when the subjects wearing left–right reversing goggles pick up an object located in the left visual field, they have to use their right hand which is normally located in the right visual field and whose visual information is normally projected to left V1 (ipsilateral to the visual field in which the object appears). To perform such actions, the subjects [...] must change their strategic (top-down) perceptual-motor control. It is well-known that top-down visual imagery alone can activate primary visual cortex with a topographical representation [...]. In addition, there are several pieces of evidence which suggest that tactile and proprioceptive perception involve visual imagery and activate visual cortices [...] 


Hence, even when neural adaptation occurs in the visual cortex, it is mainly motor, tactile and proprioceptive adaptations playing a role in adjusting to visual shifts and not so much gross changes in the visual field of view.            

References
- Harris, Psychological Review (1965); 72(6): 419-44
- Hatada et al., Exp Brain Res (2006); 173(3): 415–24
- Hatada et al., Exp Brain Res (2006); 174(1): 189–98
- Hatada &amp; Rosetti, Exp Brain Res (2006); 169(3): 417–26
- Miyauchi et al., J Physiol - Paris (2004); 98: 207–19
- Richter et al., Exp Brain Res (2002); 144(4): 445–57
- Sugita et al., Nature (1996); 380: 523–6