As a graduate student, am I allowed to use university letterhead in a letter of recommendation for an undergraduate student?
Yes, letters of recommendation are official university business, so you can use letterhead.  It would be unusual if you did not.