Understanding math behind RSA key derivation
Well, if $e$ is not relatively prime to $\phi(N)$, then there won't be any such $d$; in fact, there generally won't be a unique decryption.

For example, let us consider the toy example $N = 77$ and $e=3$.  Note, that $\phi(N) = 6 \times 10 = 60$ is not relatively prime to $e$.

Then, let us assume that we receive a ciphertext $41$, and we need to find the plaintext $P^3 \equiv 41 \pmod{77}$.  The problem is that there are three; we have $P=13$; as $13^3 \bmod 77 = 41$.  However, we also have $P=24$; as $24^3 \bmod 77 = 41$ as well.  pick a plaintext $P=13$ and compute the corresponding ciphertext $C=13^3 \bmod 77 = 41$.  $P=68$ also works.  So, which is the right one?  The decryptor has no way of knowing.

We avoid this problem if $e$ and $\phi(N)$ are relatively prime.