Crawl space insulation
You can seal off the exterior vent if you are adding a fan that pushes air in and out of the crawlspace (air circulation, mimicking what that outside vent does).  However, those vents are passive and require no power, so why not leave it open and seal it off when temperatures drop below 15-20 degrees Fahrenheit?

You would not gain much from insulating the walls with a rigid foam board. Your best bet would be to insulate under your floor with an R-30 or higher fiberglass batt insulation.  That will seal your home tight with insulation from the bottom, while still allowing your crawl space to function properly with ventilation.