If a decision problem is in $P$, must finding the solution be possible in polynomial-time?
No, and the example you list is a classic example: as far as we know, factoring does not appear to be in $P$, but determining whether a number is prime is definitely in $P$.

Another example: Consider the game Hex.  Consider the decision problem: given $n$, determine whether the first player has a winning strategy for Hex on a $n \times n$ board.  There is a corresponding function problem: given $n$, find such a winning strategy.  Well, the decision problem is trivial (it is known that the answer is always "yes"), but the function problem is believed to be very hard (as far as we know).