Traffic sign recognition (TSR) is an integrated part of driver assistance systems and it remains an active research topic in computer vision today. This paper proposes a solution for TSR problem which composed of robust traffic sign image descriptor and sparse classifiers. Specifically, we outline a variant of histogram of oriented gradients (HOG), namely Soft HOG (SHOG) which exploits the symmetry shape of traffic sign images to find the optimal locations of the cell of histogram for SHOG computation. We show that our compact SHOG feature is more discriminative than HOG. Furthermore, two sparse analytical polynomial based classifiers, namely Sparse Bayesian Multivariate polynomial model and Sparse Bayesian Reduced polynomial model are introduced. The proposed sparse classifiers enable implicit feature selection and alleviate the overfitting problem. This leads to higher accuracy performance with prudent set of features. Our solution is evaluated on publicly available German Traffic Sign Recognition Benchmark (GTSRB) dataset and 16 datasets from (UCI) repository. Experiment results demonstrated that the proposed method has satisfactory result when compared to state-of-the-art methods. (C) 2016 Elsevier B.V. All rights reserved.