What could be possible features of a textual word?
That is a rather broad question. If you are on a document level, you could (after removing stop words) determine the most important words in your corpus by calculating the tf/idf measure https://en.m.wikipedia.org/wiki/Tf–idf 
Then you could define a vector over the top n words and represent every document as a vector of the relative frequencies of the top n words in that document and work on these vectors. 
On a sentence level that could work as well, on a word level obviously not. Anyway, information retrieval and computational linguistics are a rather large research area. Maybe the best way would be to start with a book that provides a good overview.

Since you asked for some more specific recommendations:

If you can get access to (or actually want to buy it), definitely this one: Speech and Language Processing: An Introduction to Natural Language Processing, Computational Linguistics, and Speech Recognition. 

You can also take a shot this: Sentiment Analysis and Opinion Mining this is probably not your focus, but it covers feature representation to some extent and is for free. 

More like a handbook with practical approaches is: Programming Collective Intelligence, look for it at Google, it is also available for free.

Otherwise if you want to look for yourself for books at a library or so, your keywords are "Computational Linguistics" and "Natural Language Processing".

Hope this helps