Why is my furnace blowing hot air outside through a PVC pipe?
It's a high efficiency gas furnace, and that's the exhaust vent. DO NOT cover or plug it!

A normal furnace relies on the stack effect, to carry hot exhaust up and out of the building. Since high efficiency furnaces extract most of the heat from the exhaust, the exhaust must be forced out. Since it takes less energy to move the cooled exhaust horizontally, HE furnaces typically exhaust to the side of a building rather than the top. As the gases are cooler, PVC, CPVC, or ABS piping can be used to carry the exhaust to the outside.

In a normal furnace you'll typically only see one motor and fan (blower), which is used to draw in cool air and blow warm air throughout the house. In an HE furnace you'll also find a draft inducer or purge motor and fan, which is used to draw in combustion air and push out exhaust.