Keep all records of employment taxes for at least four years after filing the 4th quarter for the year. These should be available for IRS review. 1  Your employer identification number.  Amounts and dates of all wage, annuity, and pension payments.