How can a cryptosystem be unconditionally secure? The definition of an unconditionally secure cryptosystem states that the cryptosystem cannot be broken even with infinitely computational ressources and time.
However, since most books define the keyspace $\mathcal K$ to be finite, then with infinite time any computational device can perform an exhaustive keysearch.

Why are perfect secrecy cryptosystems unconditionally secure? I mean, how can the one-time pad with a small keyspace be considered to be unconditionally secure (take ${\mathcal K} = \{0,1\}^n$ for some small $n$)?

Are unconditionally secure cryptosystems the same as perfect secrecy cryptosystems?

In order to define computational security, one makes the relaxations of unconditional security, that it is possible to break the cryptosystem in superpolynomial time and with neglible probability. Why are these relaxations of unconditional security? Any cryptosystem can be broken with small probability by taking a key at random, even an unconditionally secure one?
The term unconditional security was (as far as I know) coined by Diffie and Hellman in their seminal paper New Directions in Cryptography. Here is the snippet


  [... ] a system which can resist any cryptanalytic attack, no matter how much computation
  is allowed, is called unconditionally secure. Unconditionally secure systems are discussed in [3] and [4] and belong to that portion of information theory, called the Shannon theory, which is concerned with optimal performance obtainable with unlimited computation. 


The reason why a perfect implementation of one time pad (OTP) can be considered to be unconditional secure is because of Shannon's insight which may be succinctly described by
$$
Pr_{\mathbf{k}\leftarrow \mathcal{K}}\left[\mathbf{c} = \mathcal{E}\left(\mathbf{m}_0,\mathbf{k}\right)\right] = Pr_{\mathbf{k}\leftarrow \mathcal{K}}\left[\mathbf{c} = \mathcal{E}\left(\mathbf{m}_1,\mathbf{k}\right)\right], 
$$ 

i.e. given a ciphertext $\textbf{c}$, the probability that it was the encryption of some plaintext $\textbf{m}_0$ is equal to the probability that it was the encryption of another plaintext $\textbf{m}_1$; and this holds true for all possible plaintexts and ciphertexts. 

Note that it does not say anything about how big or small the keyspace is; for OTP, the only requirement is that the key be at least as large as the message. 

Thinking purely from an adversarial perspective, the content of the message can be ANYTHING but there is nothing in the ciphertext that makes one message seem more probable than the other. I have tried to explain this with a cartoon image. Exhaustively searching for the original plaintext (by brute-force cryptanalysis) is an exercise in vain because every plaintext is equally likely! The world war II pigeon discovery is a case study of this kind.

Of course, in case the adversary has some side information that makes the context more specific, the actual message (or parts of it) may be correctly guessed. But it is important to know that any such information is not leaked by the OTP at least.