Robot Orientation Estimation Based on Single-Frame of Fish-eye Image

In order to develop the steering control for collision avoidance behaviour, robot must be able to determine its heading orientation with respect to environment. Orientation can be measured by dedicated sensors or through visual features perception. In vision-based orientation estimation problem, most of approaches are making use of a matching process between pair of frames. This paper proposes a method of estimating robot’s heading orientation by using only a single-frame of fish-eye image. CIE-LAB colour space is applied to handle colour and illumination intensity change. Straight line segments are extracted from thresholded CIE-LAB image take advantage of Progressive Probabilistic Hough Transform. Angle of the corresponding line segment is measured using combination of Law of Cosines and quadrant principle. Heading orientation in yaw angle is estimated by implementing voting mechanism based on region grouping and length of perpendicular line. Some experiments are made in robot soccer field environment to compare orientation estimation system against IMU’s measurement. Discussion about the performance and limitation of the system are included in this paper.


Introduction
Collision avoidance behavior has an objective to compute motion control that free of collision with obstacles [1]. This behavior is needed in intelligent vehicles [2] and also in robotics [3]. Soccer robot is one of robotic applications that implemented collision avoidance [4]. Collision avoidance can be achieved by braking [5], steering [6], or applying coordinated of these two previous maneuvers [7]. It is necessary to address how large an angle to steer the robot away from colliding with obstacle. The degree of steering angle is determined by the orientation and position of the robot at the current time with respect to the goal and obstacle location. Orientation and position can be measured by several methods. Environmental map can be chosen to find where the robot currently headed and located. In order to provide precise information about the environment, this method needs some beacons to be fitted. When the initial robot position is not known, this approach often requires iterative mechanisms. Global Positioning System (GPS) is offered to precisely locate the robot. This approach has limitation in high cost and some constraints in indoor environment. Some dedicated sensors such as compasses, altimeters, sonar, and laser range finders supply an immediate orientation and/or position. While accelerometers and odometry provide a relative position and/or orientation. Every sensor has limitations. In case that the robot has to be operated in environment that is affected by magnetic field, compass should not be used because of it needs unbiased magnetic field to estimate the orientation. Such sensor can be substitute by another sensor such as camera that is not affected by magnetic field. Environment comprises a lot of visual landmarks which are beneficial for robot orientation estimation.
The tracking of mobile robot position and orientation was studied in [8]. Model of robot's environment was not provided in their research. Application of Hough transform in sonar sensor readings was combined with histograms to correct the robot's orientation. Each frame pairs were compared by computing their angle histogram using intersection-measurement. Convolution between angle histogram of previous and current place with equal minimum matching score was used to determine best orientation. The calibrated odometry was compared to the obtained results of their method. The average relative error of their proposed method and calibrated odometry were 2% and 6% respectively. A visual gyroscope based on single-frame was described in [9]. The priority of their research neglecting accuracy was reducing the amount of time in the computation of camera rotation. The rotation that caused by camera shaking usually change orientation of an image in the roll axis. The displacement of image features triggered motion blur. Sharp features remaining in the blurred image was exploited to compute center of rotation and magnitude of roll angle. Heading of a robot was incrementally estimated using panoramic images in [10]. These images were grabbed by the robot in outdoor and indoor environment. Comparing every pair of consecutive images was used to get the lowest distance in image space. This distance matched to rotation of the robot in opposite direction. Magnetic compass was used as reference in four experiments. In two outdoor experiments, maximum errors were about 56° with smooth and not smooth images. While in indoor, maximum error were about 35° and 50° with smooth and not smooth images respectively. An interesting technique for developing visual compass was studied in [11]. Feature was extracted by counting the transition frequencies between color classes in the image. Simple statistical features were used to achieve fast computation. This idea based on the assumption that the yaw angle of the camera was depended on the frequency pattern of color class transitions. The viewed scene recorded using steps of 5° was estimated as yaw angle. Error was increasing quickly until it greater than 45° if localization was above 2 m from the training location. Experiments in [12] were compared between GNSS/IMU ground truth and outdoor dataset with five variation in the field-of-view of fish-eye images. SIFT was utilized to estimate the relative orientation by extracting and matching between pairs of images. A sequence of 900 frames or nearly 4 minutes was used in their experiments. The longer interval between frame pairs was proportional to the error increment. Larger spacing increase in the estimated pairwise caused the orientations grossly wrong. Highly accurate orientation data could be achieved by shorter chains frame pairs.
In contrast to the other works above, this paper aims to estimate orientation of robot heading represented as yaw angle using only a single-frame of fish-eye image. This paper proposes a voting mechanism on Hough line segments that extracted using Progressive Probabilistic Hough Transform. In this paper, images of the environment are used to estimate some range of robot's orientation. Images are collected by the robot while it moves in the indoor robotic soccer field environment. Fish-eye lens on camera is used to provide a full view of the robot's surroundings in the form of fish-eye images. By using these images, robot can observe whole direction without turn around its view. Orientation is a measure of robot heading relative to an absolute reference. A starting heading orientation to the opponent's goal is chosen as absolute reference. This research uses an omni-directional three-wheeled mobile robot as locomotion mechanism. Center of camera is aligned with the center of the robot. This paper exploits the rotation of sharp edges present in the image relative to absolute reference. The amount of rotation of these edges with respect to absolute reference is percepted as the orientation of the robot.
The remaining of this paper is organized as follows. Section 2 discusses a method to estimate the orientation using voting technique. The experimental setup, the results of some experiments and 3 discussion of the proposed method are described in Section 3. Finally, conclusion is presented in Section 4.

Estimating the Orientation Using Voting
For detecting line segment, this paper explores CIE-LAB image as input. This type of color space is chosen to handle color and illumination intensity change. There are several steps need to be executed for detecting line segments by applying Hough Transform as depicted in figure 1. After converting RGB to CIE-LAB, it need to be converted to grayscale image. This image is used as input for the third step, thresholding. Thresholding acts as the filter for binarization using predefined threshold value for producing bi-level or binary image. Black pixels are generated from pixels with intensities below threshold value. While white pixels are resulted from pixels that have intensities higher or equal than threshold value. Our experiment uses 176 as threshold value. Image that contains only two possible value, black and white, is used as input for edge detection step. Canny algorithm is used to detect some edges from binary image. This algorithm finds the edges using two level of threshold. This experiment uses 50 and 200 as the smallest and the largest threshold value respectively. The smallest value is used for linking the edge. While the largest value determines the initial segments of strong edges. Each pixel in binary image is compared with these threshold values. The pixel is kept as edge if it has greater or equal value compared to the highest threshold. Although it has the value that is greater or equal to the lowest threshold, it can be kept as edge pixel only if there is at least one neighbouring pixel of eight neighbour cells that has the value that greater or equal to the highest threshold. After detecting edges, the next step is implementing Hough transform to binary image.    approach, SHT, can detect no line, this paper proposes to estimate the orientation of the omnidirectional three-wheeled mobile robot using detected line segments by PPHT. In this second approach that based on probabilistic algorithm, slope angle θ must be determined by using begin and end points of each line segment. Figure 2 at the left part shows a line segment that begin with (x1,y1) and end with (x2,y2). Normal line ρ is drawn from origin O intercept perpendicularly with this line segment at point C (xC,yC). This line segment can be lengthen until it cross the XI axis in point B. By connecting origin O, interception point C and intersection point B, a triangle consists of side OC, OB and BC is contructed. The OCB triangle contains θ that can be useful for orientation estimation. By utilizing law of cosines, this slope angle can be computed if magnitude of each triangle side is known. General form of law of cosines is defined in (1). For simplicity in computing slope angle θ, law of cosines in (1) is modified into (2). The magnitude of each triangle side can be calculated using (3).  Figure 2 at the right part displays the quadrant principle which divides image area by four region. From positive XI axis clockwise, there are region I, II, III and IV. Regional division is used for determining real value of slope angle θ corresponds to certain line segment. Angle θ that corresponds to a Hough line segment has real minimum value greater than 0° and real maximum value less than 180° if the intersection point C is found in region I or II. In the case C is located in these upper region, the real value of slope angle θ can be easily computed using (3). For the opposite case, in which the interception point C is discovered in region III or IV then angle θ has real value between 180° and 360°. In the case C is situated in the lower region, the real value of slope angle θ is equal with the explementary angle. This angle can be calculated by finding the difference between full circle angle 360° to slope angle θ (4).
For the second approach for orientation estimation, PPHT that proposed by this research is using average value of θ of each line segment as expressed in (5). This method is used in this research for comparison to our proposed algorithm to estimate the orientation using voting mechanism. For the third approach, our voting method for orientation estimation based on single-frame can be described by the following algorithm.

Experimental Results
In the context of visual perception, slope angle θ must be resulted from orientation estimation system for supporting motion control. For visual perception requirements of orientation estimation system, robot is equipped with a Logitech 920 web camera with a fish-eye lens pointed downwards which gives circular images. To provide position and orientation measurement, robot is prepared with rotary encoder and gyro. Two pairs of 200 ppr rotary encoder is utilized as odometry for position measurement. MPU 6050 is employed as gyro for orientation measurement. An international standard soccer robot field is managed as environment to test this orientation estimation system. An operator controls manually the robot's movement to collect video data set by the omni-vision camera. There are 159 frames extracted from data set. Each frame has resolution 640 x 360 pixels. At initial pose, robot starts to move from its own goal area headed to the opponent's goal. The direction where the robot is headed defined as absolute orintation reference. Both IMU and our proposed orientation estimation system use this direction as reference. Robot movement is started in the middle of our own goal area. Robot moves forward 0.5m/s with 0° orientation in area 1. When it arrives at area 2, robot rotates 45°/s from 0° to 360° unclockwise. Next, robot continues to move forward by 0° orientation in area 3. SHT, PPHT with implementation of law of cosines (2) and average of estimated orientation (5); and PPHT with addition of quadrant principle (4) and voting mechanism are three approaches that used in experiments on area 1, area 2 and area 3. IMU's angle measurements provides ground truth to test these approaches. SHT and second approach (PPHT_Avg) result similar respons on all areas but differ in the number of void. These void data are caused by undetected of lines or line segments as the basis of this methods. For straight motion, the first two approaches, SHT and PPHT_Avg, can not accurately estimated the orientation. While the third approach, PPHT with voting mechanism (PPHT_Voting) can estimate the orientation with 78.33% of accuration. These occur because of the limitation of algorithms (3). PPHT_Voting gives solution for these drawback by combining law of cosines (2) with quadrant principle (4) and voting mechanism. The result of the third approach is ability to estimate orientation from 0° to 360°. The accuracy of PPHT_Voting is 69.88% for area 2 and 68.75% for area 3 respectively.

Conclusion
This paper has presented a method to estimate the full-range orientation of a robot using a singleframe of fish-eye image with respect to an absolute reference in the environment. Hough line segments that are extracted from CIE-LAB images by utilizing Progressive Probabilistic Hough Transform and computed by Law of Cosines and quadrant principle with voting mechanism effects the computation time and range of orientation. This method has a potential to be developed with tracking system. Yaw angle estimation from line segments can be computed fairly consistently. Improving the accuracy and implementing it for steering control of collision avoidance behaviour are subjects of our future work.