{'fraction_non_alphanumeric': 0.019091463001827905, 'fraction_numerical': 0.00419741385146571, 'mean_word_length': 4.6902927580893685, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Sensors Fusion based Online Mapping and Features Extraction of Mobile Robot in the Road Following and Roundabout

A road feature extraction based mapping system using a sensor fusion technique for mobile robot navigation in road environments is presented in this paper. The online mapping of mobile robot is performed continuously in the road environments to find the road properties that enable the robot to move from a certain start position to pre-determined goal while discovering and detecting the roundabout. The sensors fusion involving laser range finder, camera and odometry which are installed in a new platform, are used to find the path of the robot and localize it within its environments. The local maps are developed using camera and laser range finder to recognize the roads borders parameters such as road width, curbs and roundabout. Results show the capability of the robot with the proposed algorithms to effectively identify the road environments and build a local mapping for road following and roundabout.


Introduction
The autonomous mobile robot navigation in urban building still a challenging subject for researchers worldwide, due to several uncertainty conditions that prevail during road navigating. The mobile robot isn't used just as a transportation medium but nowadays, it is attempting to utilize it for doing some services on the roads, like painting the road marks, grass cutter and cleaning…etc. A complete navigation algorithm that can consider and deal with all conditions that are suffered in the roads environment still not build yet [1,2]. One of the important cases in road environments is a navigation and path finding in the open space area eg. Roundabout, T ,Yand cross junctions. Three challenges can be happened during navigation in such area: first is the capability for detection of the printed marks during weather changes secondly is the ability to discover and analysis of the light traffic signals, the third is detection of the mentioned intersections when the printed marks are missed by using natural landmarks (borders, edges). Several researches have been touched on the road following and road intersections areas eg. T,Y and cross junctions by developing the local maps for the environments using suitable sensors. But none of them has deal with the roundabout intersection, which will be studied in this paper. The navigation in the roundabout is a challenge problem due to the following reasons: road following navigation on the road at the entrance and exit of the roundabout; robust detection for the roundabout with capability to distinguish between it and the other circular obstacles; path planning and localization of the robot in the open space area of the roundabout; on-line making decision of the exit direction to choose the proper exit of the roundabout. We will highlight here some works that touched on navigation in T,Y and cross junctions, however the roundabout will be modeled and studied in details in this paper: SCARF robot with color vision system was considered as the first system for online intersection detection [3]. It can recognize the roads with faded surfaces, edges and Y intersections without prepredictions from the navigation system. It includes two functions: detection of road surface using Bayesian networks and pattern recognition and interpretation generation using match-filter technique. A Color camera is used for road following system YARF [4] based on road features extraction. The feature trackers approach is used to enable the system to distinguish between different colors marks, and to detect if these marks are broken or continuous based on frequency absences of the feature to estimate the intersections and branches. A Camera was used with road marking analysis system (ROMA) for detecting the lane of roads and road intersections [5]. Two algorithms were used: Contour-based methods are used for road border detection; and segmentation of road to find dashed and solid marking lines for intersection detection. A Virtual camera based on artificial neural networks is used for driving the ALVINN vehicle in different types of roads like paved roads, interstate highways and road junctions [6]. It's detection depends on the color image preprocessing for a gray-scale image, 3layers neural network with backpropagation algorithm and Input Reconstruction Reliability Estimation (IRRE) method. GPS is combined with odometry and LRF for trajectory tracking, obstacle avoiding and localization in curbed roads [7]. The autonomous navigation is performed in the curbed road environment through a combination between DGPS and odometry with extended Kalman filter to localize the mobile robot within road environment; however LRF is used for obstacle avoiding when they occurred by finding the suitable path to pass in relation with roads curbs. Several works the Corr. author touched with path planning, localization and navigation in road roundabout as the first work has discussed this road area in details [8][9][10][11][12][13]. In this work, a sensor fusion technique involving camera, LRF and odometry is used to enable the robot to determine its position within the road roundabout environments. The path planning modeling of the roundabout environments is derived firstly and later implemented via an algorithm for real-time sensor fusion based navigation system. Experimental work is accomplished to validate the proposed algorithm in indoor and outdoor applications, which show the ability of the robot to navigate autonomously in the roundabout environments.

Platform Overview
A new mobile robot platform has been developed and fabricated in our Lab. The platform includes three main units as in Figure.

Sensor based Road Detection and Navigation
The robot is autonomously navigating in the curbed roads following and rotates effectively in a standard roundabout. It can find the path starting from the pre-defined start position until reaching the pre-goal points using the local map identified by camera, the laser range finder and odometry measurements. Local maps of the environments have been developed using camera, laser range finder and encoders. The data coming from each sensor are prepared in such way that can extract the features of the road as follow:

Camera
A WiFi camera is used to capture the scenes of the environments located in-front of the robot with suitable resolution (760×320) and suitable speed 30 frame/s (fps). The video sequences were processed using MATLAB as it is involved the suitable image acquisition and processing toolboxes to perform on-line capturing and processing of the video.
The captured video was processed as sequences of images that were later passed through the image processing techniques. The recovery of the proper perspective from the image is very important, due to the fact that the image data is used for the calculation of the position of robot. Normally, the object in the scene that has been captured through the images is scaled and inverted horizontally and vertically. If the object is located at a distance D from the lens, it is projected onto the image plane that is located at distance I cam from the lens. The distance between the image plane and lens is called the focal length.
The magnification factor can be defined as follows: Figure. 2 shows the elevation view of the camera installed on top of the robot. The camera is inclined at a small angle to focus on the area located in-front of the robot, where its plane represents the object plane that is reflected to the camera. For the purpose of calculation, one considers the part of the ground plane is represented by w and v in the image plane. The centerline of the camera intersects with the ground plane at point E with an angle. Two planes can be defined for the image, one is the camera image plane and the other is scaled image plane which is parallel to the camera image plane but with real dimension. The scaled image plane intersects with the ground plane at point E. One can calculate the dimensions of the scaled image plane with respect to the camera image plane by multiplying it with a magnification factor as follows: The same distance at the camera image plane v can be calculated at the ground plane using the following equations: One can apply the Equations 4 and 5 to compute the real world dimension of each pixel.
In the camera image plane, the magnitude of v and w can be considered as the positive and negative values of y i , respectively. The position of any point on the ground plane y g can be calculated from its projection in the image plane y i. as follows: Where y g is the vertical distance in the ground plane that has been computed from the knowledge of the coordinate system at the image plane. In the ground plane, one can find that the horizontal distance of the road near to the camera looks smaller than the far one. Actually, Q and S (as shown in Figure. 2 (b) have the same length in the image plane, and they still have similar length in the scaled image plane Qʹ, Sʹ, but on the ground plane, they are different due to the camera distortion, denoted by Qʹʹ, Sʹʹ. One can define following equation for x direction: From Figure 2 (b), u q and u s can be calculated as: can be calculated as:  (10) These calculations involve the scale and shift processes that have been applied to the image of the ground plane. It produces various kinds of distortions in the image, which need further correction and calibration.

Laser Range Finder
LRF measurements are used to localize the robot in environments and for building 2D local map. As previously mentioned, this device can scan an area with 240 0 at 100 ms/scan. Two coordinate systems can be used to describe the LRF measurements as shown in Figure. 5, namely, the laser measurement coordinate system and the environment coordinate system. The fact that the laser measurement in each scan is coming from the same origion, the right angle triangle calculation is used for laser measurements calculation as shown in Figure.
Where L a , L b and L c are the length of laser measurements at point a, b and c, respectively as shown in Figure. 5. β = (0, 240 0 ) is the angle between the measurement point and the platform coordinate system. y La , y Lc represent the parameters of the road and y Lb represents the curb. It is found that y La and y Lc has the same length in y direction; however the length of y Lb is different and can be written as follows: Where Z Rb = h c , h c is defined as the height of the curb which can be invoked as a threshold value in this program. ρ is the angle between the ground and the laser measurement that is known and can be adjusted in the platform. For detecting the obstacles, one can compare the sequences of the two scans measurements between levels i and ii in y L direction in the area located between d and e, as shown in Figure. 5 to detect if there is any obstacle in-front of the robot as expressed in the following equations: x e = L e cos() The width of the obstacle can be calculated as: From previous calculation, one can define three parameters that will be used later for road discovering by LRF as shown in Figure.  Where r n is the dimension of LRF signals for n-th LRF measurements.
-Curb threshold: rf 0 in β= 0 0 is used as reference and the other fluctuation measurements (r fn ) on the left and right side in the same scan are compared with this base line. If the deviation between the reference point and other measurement values exceeds the pre-defined threshold t h , this point will be considered as a road curb. Otherwise, it will be considered as a road. This operation is repeated with all measurements as follows:

Odometry
Tow rotary encoders were used to estimate the position of robot, which are connected to the differential wheels. The full rotation of encoders is 500 pulses/rotation and the linear position of the wheels is calculated from the encoder rotation as shown in Figure. 7 using the following expression: Where C is accumulative path of robot, r is the radius of wheel, P cur is the number of pulses in the current position and P fr is the number of pulses for one full rotation of the encoder.

Figure. 7 Calculation of the linear position from the rotary encoder
Two encoders were used in the proposed system and the accumulative path will be calculated as an average value of both encoders as follows: The odometry measurements are used with LRF for localization of the robot in the environments.

Video and image Processing
Video can be captured by a camera with suitable resolution, and then processed using MATLAB as it is involved the suitable Image acquisition and processing toolboxes to perform online capturing and processing of video. In general, the image sequences processing algorithm consists of three main parts: (i) Pre-processing of the image for depth processing.
(ii) Processing of the image and development of the environments local map.
(i) Pre-processing of the image for depth processing: This algorithm is used for identifying the road environment from the video and preparing the images sequences in a useful way to be used at the depth processing step. The main operations that have been used at this step can be briefly described with their MATLAB code as follows:

LRF Signal Processing
The LRF driver for reading the data from USB connection in real-time is developed in MATLAB. It includes some functions to identify, configure and get data from the sensors as further detailed out in Appendix B. The algorithm for detecting the curbs of road based on the previous mentioned Equations. 11-18 is developed and implemented in the road environments as shown in Figure. 6. The developed local map for laser range finder can be determined as shown in Figure.

Conclusions
The autonomous path navigation of mobile robot has been derived for the road following and roundabout using sensor fusion. The platform is combined with camera, encoders and LRF for sensing, DC brush motor for actuating with smart driver and IFC cards. The sensors fusion is used for simultaneous developing of the robot path and building accurate local map. The road features such as the curbs, borders; width length has been detected and measured with the sensor fusion system. Experimental results for both roads following and roundabout with camera and LRF sensors have been reported, which present the capability of the suggested algorithms to enable mobile robot to extract the road features effectively in the road environment.
Future work is to implement of the feature for road path planning and localization. Also the signal stochastic and probabilistic methods like Kalman filter is needed to eliminate the noises and improve the path of robot.