Developing an automatic weeding system requires robust detection of the exact location of the crop to be protected from damage. Computer vision techniques can be an effective means of determining plant location. In this paper, a novel algorithm based on colour features and morphological erosion and dilation is proposed. This process segments cauliflower crop regions in the image from weeds and soil under natural illumination (cloudy, partially cloudy, and sunny). The proposed algorithm uses the HSV colour space for discriminating crop, weeds and soil. The region of interest (ROI) is defined by filtering each of the HSV channels between certain values (minimum and maximum threshold values). The region is then further refined by using a morphological erosion and dilation process. The moment method is applied to determine the position and mass distribution of objects in video sequences, as well as to track crops. The performance of the algorithm was assessed by comparing the obtained results with those of ground truth methods (manual annotation). A sensitivity of 98.91% and precision of 99.04% was achieved. (C) 2016 Elsevier B.V. All rights reserved.