Dry eye is an increasingly common disease in modern society which affects a wide range of population and has a negative impact on their daily activities, such as working with computers or driving. It can be diagnosed through an automatic clinical test for tear film lipid layer classification based on color and texture analysis. Up to now, researchers have mainly focused on the improvement of the image analysis step. However, there is still large room for improvement on the machine learning side. This paper presents a methodology to optimize this problem by means of class binarization, feature selection, and classification. The methodology can be used as a baseline in other classification problems to provide several solutions and evaluate their performance using a set of representative metrics and decision-making methods. When several decision-making methods are used, they may offer disagreeing rankings that will be solved by conflict handling in which rankings are merged into a single one. The experimental results prove the effectiveness of the proposed methodology in this domain. Also, its general purpose allows to adapt it to other classification problems in different fields such as medicine and biology.