Is this a tried alternative to word embedding for NLP? I'm searching for research related to my idea, but apparently cannot articulate it well enough to the search engines to show me what's been published on this.
My idea: in a deep learning context (text classification), instead of inputting the text as integer vectors representing words in a vocabulary, which get substituted with dense vectors (word embeddings), what if instead, you represent each phrase almost like how a vector of &quot;pixels&quot; represent an image (then use 2d convolutions)? For example, each character, digit, and symbol get an integer value, just like how pixels are represented. So abc becomes 0, 1, 2. The phrase could be formatted as a rectangle at word boundaries or something, but the gist of it, a phrase is formulated as a 2d array where each element is a character from some character set.
Is this a known/tried approach, and what name does it go by? From its apparent lack of popularity, I assume this formulation is far worse than word embeddings, but I'm curious about it.
It has never been tried as far as I know. And the reason is that, IMHO, it would give you inconsistent results. I am talking in particular about unwanted arithmetic properties of such vector.
Do you want to represent an a, b, c vector of letters {1,2,3} knowing that
a &lt; b &lt; c  ?
This makes sens with ints, not with char/tokens.
The good news is you can still use 2d conv, once you assembled embedding vectors into larger phrase vectors.