Proof Review: Integer Factorization is in NP
You are missing something. 

If you are given what is supposed to be a factorisation of a number x, it's not enough to show that the product of those numbers is x. You also have to prove that all the numbers in the purported factorisation are primes. 

Fortunately there is theorem that for every prime number, there is a polynomial time proof that it is a prime.