I am working on the following problem: 

Let $A$ be a $4 \times 4$ matrix with entries in a field of characteristic zero. Suppose that $A$ commutes with both $\begin{pmatrix} 1 & 0 & 0 & 0\\ 0 & 2 & 0 & 0\\ 0 & 0 & 3 & 0\\ 0 & 0 & 0 & 4 \end{pmatrix}$ and $\begin{pmatrix} 0 & 0 & 0 & 1\\ 1 & 0 & 0 & 0\\ 0 & 1 & 0 & 0\\ 0 & 0 & 1 & 0 \end{pmatrix}$. Prove that $A$ is a scalar multiple of the identity matrix. 

I know that $A$ is a scalar multiple of the identity matrix if and only if $AB = BA$ for all other possible $4 \times 4$ matrices $B$ with entries in a field of characteristic $0$. However, I'm struggling with deducing here that $A$ commuting with *these specific* matrices forces $A$ to be a scalar multiple of the identity matrix. Does commuting with these specific matrices force $A$ to commute with all $4 \times 4$ matrices with entries in a field of characteristic $0$? If so, how can I deduce this? 

Thanks!