CNN - Is this a Toeplitz Matrix?
First of all, be assured that the kernel matrix (the 2x2 matrix in the figure) in CNN is not constrained to have its diagonal filled with a unique value, and is thus not a Toeplitz matrix. 

In www.deeplearningbook.org,  we can read: 


  "Discrete convolution can be viewed as multiplication by a matrix, but the matrix has several entries constrained to be equal to other entries"


This means that the global operation of passing a kernel on the input data of a CNN could be expressed as the multiplication of this input data by a matrix. i.e. by a large and sparse Toeplitz matrix.

Authors mention the illustrative simple case of univariate discrete convolution. In univariate discrete convolution, we would be applying a 1-D kernel matrix of length $m$ on 1-D input data of length $n$. Let us take n=10 and m=3.
Doing the convolution could be done by multiplying the input vector data by a n*n matrix, whose diagonal would be composed of the 3 weights of the kernel, shifted of one column to the right at each new row.