Is there a limit on the number of orphan transactions a node can cache?
The 0.6 release includes fixes for a couple of denial-of-service vulnerabilities, including this one (see commit 142e6041).

Disconnecting peers that send "too many" orphans might be implemented in a future release.

In the future, if you find a DoS or security vulnerability in a Bitcoin implementation it is best to email the developer or developers so they know about the problem as soon as possible.