LabelEncoder with a Multi-Layer Perceptron?
Your colleague was correct that LabelEncoder is meant for the target variable (a.k.a. labels); more-or-less equivalent OrdinalEncoder is meant for the independent variables.

IMO, your colleague is partially correct about an ordinal encoding "not making much of a difference" for a neural network.  A neural net can contort its way into a correct understanding of your labels anyway, although I think an ordinal encoding will make its job substantially more difficult, so I still wouldn't advocate for it.  The intervening steps, scaling and PCA, make it less clear exactly what the neural network will actually see, and I don't know if that alleviate or worsen the issue of an ordinal encoding.  (Given your good score, perhaps alleviate.)