Getting value of 2 in normalized confusion matrix
Switch the axes:
# Build the table
confusion_table = pd.crosstab(df[&quot;prediction&quot;], df[&quot;labels&quot;])

# Normalise
confusion_table = (confusion_table / consufion_table.sum(0))

# Verify
print(confusion_table.sum())

Alternatively, you can play with confusion_table.divide and the axis argument: https://pandas.pydata.org/pandas-docs/stable/reference/api/pandas.DataFrame.divide.html