We consider how to privately determine whether a private set owned by Bob is a subset of another private set owned by Alice. This problem has many applications in online collaboration. We first propose an encoding method to encode a set to a vector, which can reduce a set computation problem to a vector computation. Based on this encoding scheme and two different homomorphic encryption schemes, we present two efficient protocols for private subset problem in case where both private sets are subsets of a known universal set. These protocols are secure both in the semi-honest model and in the malicious model. We then show how to use these protocols to privately determine whether a private number is a factor of another private number. (C) 2017 John Wiley & Sons, Ltd.