Gesture recognition with Hidden Markov Models I am currently working on a gesture recognition system (for an Android Application). I think that I have completed the Image processing stage, where I am able to extract the contour of the hand (I am wearing a glove to avoid background subtraction for now).

I am also getting the bounding ellipse/rectangle, the centroid as 'important' shape features of the hand.

My problem is that not a lot of literature indicate what the next stage is before the classification of the image through the hidden markov model. I am finding this 'Feature Extraction' stage very ambiguous. 

Currently I am getting a list of angles of the contour (which is approximated so as to obtain a limited number of angles)

The problem is that I am clueless as what to do next. When I search for literature as 'Shape classification using HMMs' I still find it hard to what should I do next.

As a tool for HMM I plan to use JaHMM. But I don't know how to experiment with the tool at this stage since I do not know what will be the input to this classification stage!



I have encountered a list of functions I found in some literature, such as Fast Fourier Descriptors, Curvature Descriptors, B Spline; yet I am clueless as to how to apply these functions to my current data (i.e. list of angles, e.g.: -63, 154, 3, 23, 54, ....)



UPDATE 1:

Thank you for your information. @Peter K.

Regarding the poses: I was going to go for a set of words in American Sign Language which are fingerspelled, e.g 'dog' -> 'd' 'o' 'g' (3 states); but the moment I haven't decided what to go for.

I am currently reading some more papers to see what type of information I should extract, such as :


centroid of hand
angle of motion
distance from a particular point to different points of the hand contour (etc..)


Now I have encountered a paper which seems to show what I want to do, I'm not sure:

http://espace.library.uq.edu.au/eserv.php?pid=UQ:10700&amp;dsID=n0273.pdf

I am currently reading section 5 - Vector Quantization (I heard of this term before but do not actually know what it involves, and in figure 5 there seems to be a complex algorithm which, if I understood correcty, converts the set of values I achieve from the hand (just mentioned) into 1 digit which I can use as the Observation sequence to train an HMM for 1 particular sign. Do you think I'm moving on the right track? (I'm working on Android, (NDK), I found JaHMM as an HMM tool, and using OpenCV for image processing.

@Peter K. Thank you for your answer. Regarding the data generation I was planning to follow the steps of this paper, after I produce my personal dataset:
(section 4 and 5)
http://www.i.ci.ritsumei.ac.jp/~shimada/papers/vi02-tanibata.pdf 



UPDATE 2: keeping in mind that a gesture is formed of {posture a, posture b, posture c}

I am now thinking that I must make use of some form of classification algorithm. That is, I currently have a set of feature vectors:

Posture A:
[angle of ellipse surrounding it, height:width ratio]


0.802985 33.909615
0.722824    31.209663
0.734535    30.206722
0.68397 31.838253
0.713706    34.29641
0.688798    30.603661
0.721395    34.880161


Posture B:
[structured the same as posture A]


0.474164 16.077467


0.483104 14.526289
0.478904 14.800572
0.483134 14.523611
0.480608 14.41159
0.481552  15.563665
0.497951  15.563585



etc..

and I would like that when I feed a feature vector I obtain a simple symbol, e.g. 'A', 'B', etc.

Is this possible? I also migrated the question here: https://stackoverflow.com/questions/15602963/vector-quantization-algorithms-used-to-provide-observation-sequences-for-hidden
I have used HMM for gesture recognition (not pose recognition). What I did was: tracking the hand and recognize the gesture the hand was drawing in the air, you can image it as a trail.

You can use HMM as sequence recognizer, so first of all you need to transform your image into a discrete number sequence.

For each gesture you want to recognize, you need to train an HMM for that. 

So you have a dictionary with some known word. Each one is a trained HMM. If you have a new word (unknown obdervation) you can compute the probability for each word of the dictionary to be likely the unknown one.

Some pseudo-code:

##################### training phase

examples = [112233, 11233, 123, 1122223333]
word1 = train(example)

examples = [222333, 22222223333, 2222333333]
word2 = train(example)

examples = [124555, 1122445, 1111111222224444555]
word3 = train(example)

dictionary = [word1, word2, word3]


##################### recognition phase

#let's say you have a new unkown word: 12245, you want to know what word of the dictionary it is more likly

unkown = 12245
probabilities = []
for w in word:
    probabilities.append( calculate_prob(unkown) )


Now, see what the is the maximum value in probabilities and you get the most likely word of the dictionary!

See here:

http://www.ece.ucsb.edu/Faculty/Rabiner/ece259/Reprints/tutorial%20on%20hmm%20and%20applications.pdf

http://www.codeproject.com/Articles/69647/Hidden-Markov-Models-in-C

http://www.creativedistraction.com/demos/gesture-recognition-kinect-with-hidden-markov-models-hmms/