use hidden layer of word2vec instead of 'one-hot', to reduce number of weights in other nets? I've been reading about word2vec and it's ability to encode words into vector representations. Coordinates (probabilities) of these words are clustered together with their usual context-neighbor words.

For example, if we have 10k unique words in our dataset, we will feed 10 000 features into the network. The network can have for example 300 hidden neurons, each with linear activation function. 

The output of the net are 10 000 neurons, each with a softmax classifier. Every such a "soft-maxed output" represents probability to select the appropriate word from our 10k dictionary.


Question 1: Well, I was able to get benefit of "semantically grouped" words via that final softmaxed-output layer. However, it seems way too large - it still has dimension of 10 000 probabilities.

Can I actually use that 300 neurons result in some other networks of mine? Say, in my LSTM, etc? 

Then weights of my LSTM wouldn't take so much space on disk. However, I would need to decode the 300-dimensional state back into my 10k-dimensional state, so I can look it up

Question 2: ...are the "Encoded" vectors actually already sitting in the Hidden layer or in the Output Layer? 

References:
first, second, third
Are you asking if the 300 dimension 'middle layer' has learned to encode words into an embedding like word2vec? 

If this is the case, I'd say no. There's nothing in the one-hot vector that specifies distance between two words. For instance, the one-hot vectors for cat and table will be the same distance away for cat and lion. With embeddings, clearly you want to have the second example closer together.