Do we put all the transactions in the memory pool? I wonder if orphan transactions are also stored in the memory pool? If yes, how do the Bitcoin client distinguish between the orphan and non-orphan transactions?
Bitcoin Core maintains a separate orphan pool, with transactions whose parents are missing. Its main purposes is dealing with peers that send us groups of independent transactions in the wrong order, so this pool does not need to be large.

Since Bitcoin Core v0.13.0, all transactions sent to peers are sent in batches, and within those batches in dependency order. This massively reduced the average size of orphan pools on the network (from hundreds to typically just a few).