Abstract To optimize fruit production, a portion of the flowers and fruitlets of apple trees must be removed early in the growing season. The proportion to be removed is determined by the bloom intensity, i.e., the number of flowers present in the orchard. Several automated computer vision systems have been proposed to estimate bloom intensity, but their overall performance is still far from satisfactory even in relatively controlled environments. With the goal of devising a technique for flower identification which is robust to clutter and to changes in illumination, this paper presents a method in which a pre-trained convolutional neural network is fine-tuned to become specially sensitive to flowers. Experimental results on a challenging dataset demonstrate that our method significantly outperforms three approaches that represent the state of the art in flower detection, with recall and precision rates higher than 90 . Moreover, a performance assessment on three additional datasets previously unseen by the network, which consist of different flower species and were acquired under different conditions, reveals that the proposed method highly surpasses baseline approaches in terms of generalization capability.
@cite_41: Abstract The overall goal of this study is to develop an effective, simple, aptly computer vision algorithm to detect and count citrus on the tree using image processing techniques, to estimate the yield, and to compare the yield estimation results obtained through several methods. This new citrus recognition and counting algorithm was utilized the color features (or schemes) to present an estimate of the citrus yield, and the corresponding models are developed to provide an early estimation of the citrus yield. Citrus images were taken from Jeju, South Korea during daylight and the citrus recognition and counting algorithm were tested on 84 images which were collected from 21 trees. The citrus counting algorithm consisted of the following steps: convert RGB image to HSV, thresholding, orange color detection, noise removal, watershed segmentation, and counting. Distance transform and marker-controlled watershed algorithms were evaluated for automated watershed segmentation in citrus fruits to obtain good result. A correlation coefficient R 2 of 0.93 was obtained between the citrus counting algorithm and counting performed through human observation. The proposed algorithm showed great potential for early prediction of the yield of single citrus trees and the possibility of its uses for further fruit crops.
@cite_9: Crop yield estimation is an important task in apple orchard manage- ment. The current manual sampling-based yield estimation is time-consuming, la- bor-intensive and inaccurate. To deal with this challenge, we developed a comput- er vision-based system for automated, rapid and accurate yield estimation. The system uses a two-camera stereo rig for image acquisition. It works at nighttime with controlled artificial lighting to reduce the variance of natural illumination. An autonomous orchard vehicle is used as the support platform for automated data collection. The system scans both sides of each tree row in orchards. A computer vision algorithm detects and registers apples from acquired sequential images, and then generates apple counts as crop yield estimation. We deployed the yield esti- mation system in Washington state in September, 2011. The results show that the system works well with both red and green apples in the tall-spindle planting sys- tem. The crop yield estimation errors are -3.2 for a red apple block with about 480 trees, and 1.2 for a green apple block with about 670 trees.
@cite_1: This paper presents a multi-class image segmentation approach to automate fruit segmentation. A feature learning algorithm combined with a conditional random field is applied to multi-spectral image data. Current classification methods used in agriculture scenarios tend to use hand crafted application-based features. In contrast, our approach uses unsupervised feature learning to automatically capture most relevant features from the data. This property makes our approach robust against variance in canopy trees and therefore has the potential to be applied to different domains. The proposed algorithm is applied to a fruit segmentation problem for a robotic agricultural surveillance mission, aiming to provide yield estimation with high accuracy and robustness against fruit variance. Experimental results with data collected in an almond farm are shown. The segmentation is performed with features extracted from multi-spectral (colour and infrared) data. We achieve a global classification accuracy of 88 .
@cite_17: It has been suggested that apple ( Malus * domestica Borkh) flowering distribution maps can be used for site-specific management decisions. The objectives of this study were (i) to study the flower density variability in an apple orchard using image analysis and (ii) to model the correlation between flower density as determined from image analysis and fruit yield. The research was carried out in a commercial apple orchard in Central Greece. In April 2007, when the trees were at full bloom, photos of the trees were taken following a systematic uniform random sampling procedure. In September 2007, yield mapping was carried out measuring yield per ten trees and recording the position of the centre of the ten trees. Using this data (the measured yield of the trees and the pictures samples, representing the flower distribution), an image processing-based algorithm was developed that predicts tree yield by analyzing the picture of the tree at full bloom. For the evaluation of the algorithm, a case study scenario is presented where the error of the predicted yield was set at 18 . These results indicated that potential yield could be predicted early in the season from flowering distribution maps and could be used for orchard management during the growing season.
@cite_0: Despite extensive research conducted in machine vision for harvesting robots, practical success in this field of agrobotics is still limited. This article presents a comprehensive review of classical and state-of-the-art machine vision solutions employed in such systems, with special emphasis on the visual cues and machine vision algorithms used. We discuss the advantages and limitations of each approach and we examine these capacities in light of the challenges ahead. We conclude with suggested directions from the general computer vision literature which could assist our research community meet these challenges and bring us closer to the goal of practical selective fruit harvesting robots.
@cite_19: Addressing the challenges of feeding the burgeoning world population with limited resources requires innovation in sustainable, efficient farming. The practice of precision agriculture offers many benefits towards addressing these challenges, such as improved yield and efficient use of such resources as water, fertilizer and pesticides. We describe the design and development of a light-weight, multi-spectral 3D imaging device that can be used for automated monitoring in precision agriculture. The sensor suite consists of a laser range scanner, multi-spectral cameras, a thermal imaging camera, and navigational sensors. We present techniques to extract four key data products — plant morphology, canopy volume, leaf area index, and fruit counts — using the sensor suite. We demonstrate its use with two systems: multi-rotor micro aerial vehicles and on a human-carried, shoulder-mounted harness. We show results of field experiments conducted in collaboration with growers and agronomists in vineyards, apple orchards and orange groves.
@cite_50: Automated harvesting requires accurate detection and recognition of the fruit within a tree canopy in real-time in uncontrolled environments. However, occlusion, variable illumination, variable appearance and texture make this task a complex challenge. Our research discusses the development of a machine vision system, capable of recognizing occluded green apples within a tree canopy. This involves the detection of “green” apples within scenes of “green leaves”, shadow patterns, branches and other objects found in natural tree canopies. The system uses both thermal infra-red and color image modalities in order to achieve improved performance. Maximization of mutual information is used to find the optimal registration parameters between images from the two modalities. We use two approaches for apple detection based on low and high-level visual features. High-level features are global attributes captured by image processing operations, while low-level features are strong responses to primitive parts-based filters (such as Haar wavelets). These features are then applied separately to color and thermal infra-red images to detect apples from the background. These two approaches are compared and it is shown that the low-level feature-based approach is superior (74 recognition accuracy) over the high-level visual feature approach (53.16 recognition accuracy). Finally, a voting scheme is used to improve the detection results, which drops the false alarms with little effect on the recognition rate. The resulting classifiers acting independently can partially recognize the on-tree apples, however, when combined the recognition accuracy is increased.
@cite_31: This work details the development and validation of an algorithm for estimating the number of apples in color images acquired in orchards under natural illumination. Ultimately, this algorithm is intended to enable estimation of the orchard yield and be part of a management decision support system. The algorithm includes four main steps: detection of pixels that have a high probability of belonging to apples, using color and smoothness; formation and extension of ''seed areas'', which are connected sets of pixels that have a high probability of belonging to apples; segmentation of the contours of these seed areas into arcs and amorphous segments; and combination of these arcs and comparison of the resulting circle with a simple model of an apple. The performance of the algorithm is investigated using two datasets. The first dataset consists of images recorded in full automatic mode of the camera and under various lighting conditions. Although the algorithm detects correctly more than 85 of the apples visible in the images, direct illumination and color saturation cause a large number of false positive detections. The second dataset consists of images that were manually underexposed and recorded under mostly diffusive light (close to sunset). For such images the correct detection rate is close to 95 while the false positive detection rate is less than 5 .
@cite_10: Innovative concept proposed for apple counting in tall spindle apple trees orchard.Over-the-row system with tunnel structure provide uniform illumination for imaging.Image processing for apple identification on tree canopy images.Repetitive counting of apples from two sides was avoided using 3D information. Accurate crop-load estimation is important for efficient management of pre- and post-harvest operations. This information is crucial for the planning of labor and equipment requirement for harvesting and transporting fruit from the orchard to packing house. Current machine vision-based techniques for crop-load estimation have achieved only limited success mostly due to: (i) occlusion of apples by branches, leaves and or other apples, and (ii) variable outdoor lighting conditions. In order to minimize the effect of these factors, a new sensor system was developed with an over-the-row platform integrated with a tunnel structure which acquired images from opposite sides of apple trees. The tunnel structure minimized illumination of apples with direct sunlight and reduced the variability in lighting condition. Images captured in a tall spindle orchard were processed for identifying apples, which achieved an identification accuracy of 79.8 . The location of apples in three-dimensional (3D) space was used to eliminate duplicate counting of apples that were visible to cameras from both sides of the tree canopy. The error on identifying duplicate apples was found to be 21.1 . Overall, the method achieved an accuracy of 82 on estimating crop-load on trees with dual side imaging compared to 58 with single side imaging. Over-the-row machine vision system showed promise for accurate and reliable apple crop-load estimation in the apple orchards.
@cite_11: In apple harvesting robot, the first key part is the machine vision system, which is used to recognize and locate the apples. In this paper, the procedure on how to develop an automatic recognition vision system guided for apple harvesting robot, is proposed. We first use a color charge coupled device camera to capture apple images, and then utilize an industrial computer to process images for recognising fruit. Meanwhile, the vector median filter is applied to remove the color images noise of apple, and images segmentation method based on region growing and color feature is investigated. After that the color feature and shape feature of image are extract, a new classification algorithm based on support vector machine for apple recognition is introduced to improve recognition accuracy and efficiency. Finally, these procedures proposed have been tested on apple harvesting robot under natural conditions in September 2009, and showed a recognition success rate of approximately 89 and average recognition time of 352ms.
Aggelopoulou and colleagues presented in @cite_17 one of the first works using computer vision techniques to detect flowers. That method is based on color thresholding and requires image acquisition at specific daylight times, with the presence of a black cloth screen behind the trees. Thus, although its reported error in predicted yield is relatively low ( @math 84.3 More advanced computer vision techniques have been employed for fruit quantification @cite_0 . A multi-class image segmentation for agrovision is proposed by in @cite_1 , classifying image pixels into leaves, almonds, trunk, ground and sky. Their method combines sparse autoencoders for feature extraction, logistic regression for label associations and conditional random fields to model correlations between pixels. Some other methods are based on support vector machine (SVM) classifiers that use information obtained from different shape descriptors and color spaces as input @cite_19 @cite_11 . Compared to existing methods for flower detection, these methods are more robust since morphological characteristics are taken into account. As many other shape-based and spectral-based approaches @cite_31 @cite_50 @cite_9 @cite_41 , these techniques are, however, still limited by background clutter and variable lighting conditions in orchards @cite_10 .