What does "running in polynomial time" really mean?
Perfect secrecy is achievable in a few cases, such as one-time pads, and, well, that's pretty much it. Most cryptographic protocols are vulnerable to an all-powerful, all-knowing attacker. If you do not put any restriction on what the attacker can do, then


Guess the key.
Profit.


breaks almost any cryptography, as does


Wave a magic wand.
Profit.


So at the very least we must restrict our model attackers who can do things we haven't even imagined. Cryptography also doesn't aim to protect against lead pipe cryptanalysis. So we assume that attackers are only allowed to do things “inside the system”. Based on Turing's thesis, we assume that the attacker can only compute computable functions, and that the function computed by the attacker does not depend on the key. (It's ok if the attacker can make random choices too, as long as their probability isn't influenced by the key.)

Even a deterministic attacker who does not know the key can still break most cryptographic protocols with enough effort. All he has to do after all is enumerate all possible keys. But for sufficiently large key sizes, this should be unrealistic: as a rule we aren't interested in attacks that would take more than the lifetime of the universe.

If you want to analyse the resistance of an algorithm very precisely, you would need to know exactly how powerful the attacker's computer is, and study all possible ways in which he can make the computation required by the attack to find which one is the fastest (or more generally the cheapest). This is impractical. To start with, we look at the asymptotic behavior of the attacks: the way the attacks become harder as the parameters (such as key size) become larger.

In computational complexity, there is a nice break between polynomial and super-polynomial algorithms. This break fits cryptography well. The usual methods of doing realistic stuff tend to be polynomial, whereas enumerating all keys is exponential in the size of the key (or more precisely in the strength of the key). So restricting to polynomial attackers combines two nice properties:


It is good for getting theoretical results.
It does a good job of including realistic attackers and excluding unrealistic ones.