does dryer lint come into my bathroom through the air vent?
It's mostly a guess based on the minimal information provided, but I doubt you're seeing dryer lint. Dust fibers from clothing and toilet tissue accumulate on exhaust fan grilles. They need to be pulled down and cleaned at least annually. 

Actually, the fan blades do, too. In some cases the moisture and dust conditions are so severe that the blades get completely coated and blocked over time, such that the fan moves virtually no air.