Any process that stops, prevents, or reduces the frequency, rate or extent of the directed movement of phytosterols into the blood by absorption from the small intestine.
negative regulation of intestinal phytosterol absorption