Can Eve impersonate Alice or Bob by using a replay attack?
well, you don't really explain what happens in each step, and how the authentication procedure works, but your first suggestion is at the right direction. However, the impersonator wishes to authenticate himself, so the attack should actually be

$1. A \rightarrow E(B) : A, K_{AB}${$N_A$}
$2. E(A) \rightarrow B : E(A), K_{AB}${$N_A$}
$3. B \rightarrow E(A) : B, N_A, K_{AB}${$N_B$}
$4. E(B) \rightarrow A : E(B), N_A, K_{AB}${$N_B$}

and you are missing the last reply, but it is easy to see that Eve can generate both $N_A$ and $N_B$ and thus succeed with authenticating itself.
Try to write it more carefully: what exactly is begin sent at each step (i.e., what is the meaning of sending "A"?), and what verification each side performs at each step. 

For the second example, such an attack cannot work. The identity $A$ is included inside the encryption, and Eve has no way to replace it.