Use the output of 2 hidden neurons in the last hidden layer of a NN to visualize the result of a 4-class classification task
I'm somewhat new to the topic as well, but I think what you are looking for is an autoencoder.  I have only used it in the h2o deeplearning package, but it seems to work well.

The idea behind an auto-encoder is to begin with your inputs, encode down to less nodes (two is nice for visual representation), then decode back to your original outputs (ass accurately as possible).



If the auto-encoder can decode back to an accurate representation of your inputs, then there was sufficient important information withheld in those two nodes.  You can then plot the features that those two nodes held in a two-dimensional plot that would be analogous to a bounded PCA plot.

Also - the auto-encoder is not constrained to linearity like the PCA is.  Below is a quick snippet of some code, hope it helps.

library(h2o)
localH2O = h2o.init(ip = "localhost", port = 54321, startH2O = TRUE, min_mem_size = "3g", max_mem_size = "4g", nthreads = -1)
dat_h2o &lt;- as.h2o(train.df)

unsupervised &lt;- 
  h2o.deeplearning(x = 24:356,  # column numbers to use
                   training_frame = dat_h2o, # data in H2O format
                   autoencoder = TRUE, ## unsupervised autoencoding
                   activation = "Tanh", # or 'Tanh' 'Rectifier' 'WithDropout' node activation function, Tanh seems to work best for autoencoding
                   hidden = c(5,2,5), # three layers of nodes, with 5/2/5 nodes, respectively
                   epochs = 1) # max. no. of epochs 

## layer 1 corresponds to hidden[1], so it will reduce to 5 variables
## below is roughly similar to using predict(pca.object, newdata)
training_data &lt;- h2o.deepfeatures(unsupervised, dat_h2o, layer = 1)
testing_data &lt;- h2o.deepfeatures(unsupervised, test_h2o, layer = 1)

## explore the second layer with 2 nodes, similar to exploring PC1 vs PC2
train_supervised_features2 = h2o.deepfeatures(unsupervised, dat_h2o, layer=2)

plotdata2 = as.data.frame(train_supervised_features2)
plotdata2$label = as.character(as.vector(dat_h2o[,364]))

## L2 corresponds to layer 2, so use L2
qplot(DF.L2.C1, DF.L2.C2, data = plotdata2, color = label, main = 'Neural network: 5-2-5')