Random forest confusion matrix encountered invalid values
Most probably, sum across one of the rows is coming out as zero in this code
matrix = matrix.astype('float') / matrix.sum(axis=1)[:, np.newaxis]

It is throwing a runtime warning and only that particular cell will be np.inf. Rest all division will be fine. That's why the plot is showing some data.
 You may see the same in this sample code
import numpy as np
matrix = np.array([[1, 2, 3, 0, -6],[6, 7, 8, -4, 10]]) #Sum zero for 1st row

matrix.astype('float') / matrix.sum(axis=1)[:, np.newaxis]


/usr/local/lib/python3.6/dist-packages/ipykernel_launcher.py:4: RuntimeWarning: invalid value encountered in true_divide after removing the cwd from sys.path.
array([[        inf,         inf,         inf,         nan,        -inf],
       [ 0.22222222,  0.25925926,  0.2962963 , -0.14814815,  0.37037037]])

As a solution, you should add a constant in the denominator.1 will be best
import numpy as np
matrix = np.array([[1, 2, 3, 0, -6],[6, 7, 8, -4, 10]]) #Sum zero for 1st row

matrix.astype('float') / (matrix.sum(axis=1)[:, np.newaxis] + 1)