Why infinite sampling is not realisitc assumpition in most real applications
It is not a realistic assumption because you don't have infinite time or decimal precision to find the absolutely correct value function, but you don't need that anyway since a rough estimate of it will be enough to improve the policy.

If your question is why you can't let the agent learn indefinitely in real applications, I'm guessing it is because it may be potentially expensive or dangerous to let it explore randomly in a real scenario, so you want to deploy it with an optimal or near-optimal, deterministic, policy.