Auto-Encoder to condense (pre-process) large one-hot input vectors? In my 3D game there are 300 categories to which a creature can belong. 
I would like to teach my RL agent to make decisions based on its 10 closest monsters

So far, my Neural Network input vector is a concatenation of ten 300-dimensional one-hot encoded vectors. It would be awesome if I had, say ten 40-dimensional vectors instead.

So, is it possible to pass a 300-dimensional one-hot encoded vector through an auto-encoder, to snatch its compressed version (embedding) from the middle of the auto-encoder?

This would allow me to concatenate several of such compressed embeddings as a "total" input (ten concatenated 40D vectors), without bloating this total input vector.

In my game, each one-hot is supposed to represents a distinct category. My categories are not correlating in any way.

1) Will it be valid, or will it introduce unwanted assumptions, like Label-Encoding does?

2) How to train this Auto-Encoder? Can I simply pass my one-hots through it, and fine tune until I get an exact one-hot on the other end of the Auto-encoder?

3) Is there a better way to describe several monsters?
It might be useful to think of this in terms of orthogonality. You state that "categories are not correlating in any way", which effectively means each category should be completely orthogonal to all the others -- otherwise one would be representing some degree of correlation (or anti-correlation) between at least two categories (to the degree that those categories are not purely orthogonal). Now, the only way to get 300 orthogonal vectors is in 300 dimensional space. There is no way to compress that without losing information.

Alternatively one can think in terms of using simpler dimension reduction techniques than autoencoders. Let's consider using PCA (which is, after all, just a kind of linear autoencoder with identity activation function). What does the input data look like? Presumably we have one observation for each creature, so 300 observations. We have 300 features. The result is a 300x300 identity matrix. The singular values of that are all identically 1, so PCA to anything less than 300 dimensions is just going to fail. Adding non-linear activations and extra layers is not going to fix that, it will just make it noisier via stochastic optimization so you may get a result -- it just isn't a meaningful one.

What you need if you want to compress things is some correlation structure amongst your creatures. If you can come up with a set of properties/features that creatures might have and then score all your creatures accordingly you may be able to do some dimension reduction on that -- but dimension reduction on completely orthogonal vectors just isn't a meaningful thing.