1. A gesture recognition system, comprising: an image pick-up device, for capturing an image data containing a hand image; a template database, for recording multiple gesture templates representing different gestures, wherein the gesture templates are classified by angles and are respectively stored in gesture template libraries of different angle classes; a processor, for communicating with the image pick-up device and obtaining the image data, finding out a skin part from the image data, producing a skin edge by using an edge detection means, and then classifying the skin edge into multiple edge parts of different angle classes according to angles of the skin edge; an operation engine, having multiple parallel operation units respectively for performing template matching at different angles, wherein the edge parts of different angle classes are respectively sent to different parallel operation units for template matching, so as to find out the gesture templates most resembling the edge parts in the corresponding gesture template libraries of different angle classes; an optimal template selection means, for further selecting an optimal gesture template from the resembling gesture templates found out by the parallel operation units; and a display terminal, for displaying a gesture image represented by the optimal gesture template.