Is this a tried alternative to word embedding for NLP?
It has never been tried as far as I know. And the reason is that, IMHO, it would give you inconsistent results. I am talking in particular about unwanted arithmetic properties of such vector.
Do you want to represent an a, b, c vector of letters {1,2,3} knowing that
a &lt; b &lt; c  ?
This makes sens with ints, not with char/tokens.
The good news is you can still use 2d conv, once you assembled embedding vectors into larger phrase vectors.