Question: represent the article for finding a single-sentence tldr

Question: This work presents a modular and hierarchical approach to learn policies for exploring 3D environments. Our approach leverages the strengths of both classical and learning-based methods, by using analytical path planners with learned mappers, and global and local policies. Use of learning provides flexibility with respect to input modalities (in mapper), leverages structural regularities of the world (in global policies), and provides robustness to errors in state estimation (in local policies). Such use of learning within each module retains its benefits, while at the same time, hierarchical decomposition and modular training allow us to sidestep the high sample complexities associated with training end-to-end policies. Our experiments in visually and physically realistic simulated 3D environments demonstrate the effectiveness of our proposed approach over past learning and geometry-based approaches. Navigation is a critical task in building intelligent agents. Navigation tasks can be expressed in many forms, for example, point goal tasks involve navigating to a specific coordinates and semantic navigation involves finding path to a specific scene or object. Such tasks may need to be performed in known (already mapped) or unknown environments. Irrespective of the task or the setting, a core problem in navigation is exploration, i.e., how to efficiently visit as much of the environment. This is useful for pre-mapping in known environments, or actually solving tasks in known environments. Recent work from has used end-to-end learning to tackle this problem. Their motivation is three fold: a) learning provides flexibility to the choice of input modalities (classical systems rely on observing geometry through use of specialized sensors, while learning systems can infer geometry directly from RGB images), b) use of learning can improve robustness to errors in explicit state estimation, and c) learning can effectively leverage structural regularities of the real world, leading to more efficient behavior in previously unseen environments. This lead to their design of an end-to-end trained neural network based policy that processed raw sensory observations to directly output actions that the agent should execute. While use of learning for exploration is well motivated, casting the exploration problem as an end-to-end learning problem has its own drawbacks. Learning about mapping, state-estimation and path-planning purely from data in an end-to-end manner can be prohibitively expensive. Consequently, past end-to-end learning work for exploration from relies on use of imitation learning and many millions of frames of experience, but still performs worse than classical methods that don't require any training at all. This motivates our work. In this paper, we investigate alternate formulations of employing learning for exploration that retains the advantages that learning has to offer, but doesn't suffer from the drawbacks of full-blown end-to-end learning. Our key conceptual insight is that use of learning for leveraging structural regularities of indoor environments, robustness to state-estimation errors, and flexibility with respect to input modalities, happens at different time scales and can thus be factored out. This motivates use of learning in a modular and hierarchical fashion inside of what one may call a'classical navigation pipeline'. This in navigation policies that can work with raw sensory inputs such as RGB images, are robust to state estimation errors, and leverage regularities of real world layout. This in extremely competitive performance over both geometry-based methods and recent learning-based methods; at the same time requiring a fraction of the number of samples. More specifically, our proposed exploration architecture comprises of a learned mapper (and pose estimator), a global policy, and a local policy, that are interfaced via the map and an analytical path planner. The learned mapper, together with the pose estimator, produces free space maps from input RGB images. The global policy consumes this free-space map and employs learning to exploit structural regularities in layout of real world environments to produce long-term goals. These long-term goals are used to generate short-term goals for the local policy (using a geometric path-planner). This local policy uses learning to directly map raw RGB images to actions that the agent should execute. Use of learning in mapper provides flexibility with respect to input modality, learned global policy can exploit regularities in layout of real world layout of environments, while learned local policies can use visual feedback to exhibit more robust behaviour. At the same time, hierarchical and modular design and use of analytical planning, significantly cuts down the search space during training, leading to better performance as well as sample efficient learning. We demonstrate our proposed approach in visually and physically realistic simulators for the task of geometric exploration (visit as much area as possible). We work with the Habitat simulator from. While Habitat is already visually realistic (it uses real world scans from ; as environments), we improve its physical realism by using actuation and odometry sensor noise models, that we collected by conducting physical experiments on a real mobile robot. Our experiments and ablations in this realistic simulation reveal the effectiveness of our proposed approach for the task of exploration. A straight-forward modification of our method also tackles point-goal navigation tasks, and won the AI Habitat challenge at CVPR2019 across all tracks. Navigation has been well studied in classical robotics. There has been a renewed interest in the use of learning to arrive at navigation policies, for a variety of tasks. Our work builds upon concepts in classical robotics and learning for navigation. We survey related works below. Navigation Approaches. Classical approaches to navigation break the problem into two parts: mapping and path planning. Mapping is done via simultaneous localization and mapping (; ;), by fusing information from multiple views of the environment. While sparse reconstruction can be done well with monocular RGB images (Mur-Artal and Tardós, 2017), dense mapping is inefficient or requires specialized scanners such as Kinect . Maps are used to compute paths to goal locations via path planning (; ;). These classical methods have inspired recent learning-based techniques. Researchers have designed neural network policies that reason via spatial representations (; ;), topological representations (a; b), or use differentiable and trainable planners (; ; ;). Our work furthers this research, and we study a hierarchical and modular decomposition of the problem, and employ learning inside these components instead of end-to-end learning. Research also focuses on incorporating semantics in SLAM . Exploration in Navigation. While a number of works focus on passive map-building, path planning and goal-driven policy learning, a much smaller body of work tackles the problem of active SLAM, i.e., how to actively control the camera for map building. We point readers to for a detailed survey, and summarize major themes below. Most such works frame this problem as a Partially Observable Markov Decision Process (POMDP) that are approximately solved , and or seek to find a sequence of actions that minimizes uncertainty of maps . Another line of work, explores by picking vantage points (such as on the frontier between explored and unexplored regions (; ; ;) ). Recent works from; Savinov et al. (2018b); attack this problem via learning. Our proposed modular policies unify the last two lines of research, and we show improvements over representative methods from both these lines of work. Exploration has also been studied more generally in RL in the context of exploration-exploitation trade-off (; ; ;). Hierarchical and Modular Policies. Hierarchical RL (; ;) is an active area of research, aimed at automatically discovering hierarchies to speed up learning. However, this has proven to be challenging, and thus most work has resorted to using hand-defining hierarchies. For example in context of navigation, and design modular policies for navigation, that interface learned policies with low-level feedback controllers. Hierarchical and modular policies have also been used for Embodied Question Answering (a; ; b). We follow the exploration task setup proposed by where the objective is to maximize the coverage in a fixed time budget. The coverage is defined as the total area in the map known to be traversable. Our objective is train a policy which takes in an observation s t at each time step t and outputs a navigational action a t to maximize the coverage. We try to make our experimental setup in simulation as realistic as possible with the goal of transferring trained policies to the real world. We use the Habitat simulator with the Gibson and Matterport (MP3D) datasets for our experiments. Both Gibson and Matterport datasets are based on real-world scene reconstructions are thus significantly more realistic than synthetic SUNCG dataset used for past research on exploration ). In addition to synthetic scenes, prior works on learning-based navigation have also assumed simplistic agent motion. Some works limit agent motion on a grid with 90 degree rotations (; ; . Other works which implement fine-grained control, typically assume unrealistic agent motion without any noise . This consequently leads to another unrealistic assumption of knowledge of perfect agent pose. This is because since the motion is simplistic, it becomes trivial to estimate the agent pose in most cases even if it is not assumed to be known. The reason behind these assumptions on agent motion and pose is that motion and sensor noise models are not known. In order to relax both these assumptions, we collect motion and sensor data in the real-world and implement more realistic agent motion and sensor noise models in the simulator as described in the following subsection. We represent the agent pose by (x, y, o) where x and y represent the xy co-ordinate of the agent measured in metres and o represents the orientation of the agent in radians (measured counterclockwise from x-axis). Without loss of generality, assume agents starts at p 0 =. Now, suppose the agent takes an action a t. Each action is implemented as a control command on a robot. Let the corresponding control command be ∆u a = (x a, y a, o a). Let the agent pose after the action be p 1 = (x, y, o). The actuation noise (act) is the difference between the actual agent pose (p 1) after the action and the intended agent pose (p 0 + ∆u): Mobile robots typically have sensors which estimate the robot pose as it moves. Let the sensor estimate of the agent pose after the action be p 1 = (x, y, o). The sensor noise (sen) is given by the difference between the sensor pose estimate (p 1) and the actual agent pose(p 1): In order to implement the actuation and sensor noise models, we would like to collect data for navigational actions in the Habitat simulator. We use three default navigational actions: Forward: move forward by 25cm, Turn Right: on the spot rotation clockwise by 10 degrees, and Turn Left: on the spot rotation counter-clockwise by 10 degrees. The control commands are implemented as u F orward = (0.25, 0, 0), u Right: (0, 0, −10 * π/180) and u Lef t: (0, 0, 10 * π/180). In practice, a robot can also rotate slightly while moving forward and translate a bit while rotating on-the-spot, creating rotational actuation noise in forward action and similarly, a translation actuation noise in on-the-spot rotation actions. Sampling short-term goal Planned path Input to path planner Figure 1: Overview of our approach. We use a neural network based Mapper that predicts a map and agent pose estimate from incoming RGB observations and sensor readings. This map is used by a Global policy to output a long-term goal, which is converted to a short-term goal using an analytic path planner. A Local Policy is trained to navigate to this short-term goal. We use a LoCoBot 2 to collect data for building the actuation and sensor noise models. We use the pyrobot API along with ROS to implement the control commands and get sensor readings. For each action a, we fit a separate Gaussian Mixture Model for the actuation noise and sensor noise, making a total of 6 models. Each component in these Gaussian mixture models is a multi-variate Gaussian in 3 variables, x, y and o. For each model, we collect 600 datapoints. The number of components in each Gaussian mixture model are chosen using cross-validation. We implement these actuation and sensor noise models in the Habitat simulator for our experiments. We will open-source the collected data and the noise models, along with their implementation in the Habitat simulator. We propose a modular navigation model,'Active Neural Mapping'. It consists of three components: a Mapper, a Global policy and a Local policy. The Mapper predicts the map of the environment and estimates the pose of the agent based on the current observations and previous estimates. The Global policy uses the predicted map and agent pose to produce a long-term goal. The long-term goal is converted into a short-term goal using path planning. The Local policy takes navigational actions based on the current observation to reach the short-term goal. See Figure 1 for an overview. Map Representation. The Active Neural Mapping model internally maintains a spatial map, m t and pose of the agent x t. The spatial map, m t, is a 2 × M × M matrix where M × M denotes the map size and each element in this spatial map corresponds to a cell of size 25cm 2 (5cm × 5cm) in the physical world. Each element in the first channel denotes the probability of an obstacle at the corresponding location and each element in the second channel denotes the probability of that location being explored. A cell is considered to be explored when it is known to be free space or an obstacle. The spatial map is initialized with all zeros at the beginning of an episode, m 0 = 2×M ×M. The pose x t ∈ R 3 denotes the x and y coordinates of the agent and the orientation of the agent at time t. The agent always starts at the center of the map facing east at the beginning of the episode, Mapper. The Mapper (f M ap) takes in the current RGB observation, s t, the current and last sensor reading of the agent pose x t−1:t, last agent pose and map estimates,x t−1, m t−1 and outputs an updated map, m t, and the current agent pose estimate,x t, (see Figure 2): m t,x t = f M ap (s t, x t−1:t,x t−1, m t−1 |θ M), where θ M denote the trainable parameters of the Mapper. It consists of two learned components, a Projection Unit and a Pose Estimator. The Projection Unit outputs a egocentric top-down 2D spatial map, p ego t 2×V ×V (where V is the vision range), predicting the obstacles and the explored area in the current observation. The Pose Estimator predicts the agent pose (x t) based on past pose estimate (x t−1) and last two egocentric map predictions (p ego t−1:t). It essentially compares the current egocentric map prediction to the last egocentric map prediction transformed to the current frame to predict the pose change between the two maps. The egocentric map from the Projection Unit is transformed to a geocentric map based on the pose estimate given by the Pose Estimator and then aggregated with the previous spatial map (m t−1) to get the current map(m t). More implementation details of the Mapper are provided in the Appendix. Global Policy. The Global Policy takes h t ∈ 4×M ×M as input, where the first two channels of h t are the spatial map m t given by the Mapper, the third channel represents the current agent position estimated by the Mapper, the fourth channel represents the visited locations, i.e. ∀i, j ∈ {1, 2, . . ., m}: We perform two transformations before passing h t to the Global Policy model. The first transformation subsamples a window of size 4 × G × G around the agent from h t. The second transformation performs max pooling operations to get an output of size 4 × G × G from h t. Both the transformations are stacked to form a tensor of size 8 × G × G and passed as input to the Global Policy model. The Global Policy uses a 5-layer convolutional neural network to predict a long-term goal, g We use the Habitat simulator with the Gibson and Matterport (MP3D) datasets for our experiments. Both Gibson and Matterport consist of scenes which are 3D reconstructions of real-world environments, however Gibson is collected using a different set of cameras, consists mostly of office spaces while Matterport consists of mostly homes with a larger average scene area. We will use Gibson as our training domain, and use Matterport for domain generalization experiments. The observation space consists of RGB images of size 3 × 128 × 128 and base odometry sensor readings of size 3 × 1 denoting the change in agent's x-y coordinates and orientation. The actions space consists of three actions: move_forward, turn_left, turn_right. Both the base odometry sensor readings and the agent motion based on the actions are noisy. They are implemented using the sensor and actuation noise models based on real-world data as discussed in Section 3.1. We follow the Exploration task setup proposed by where the objective to maximize the coverage in a fixed time budget. Coverage is the total area in the map known to be traversable. We define a traversable point to be known if it is in the field-of-view of the agent and is less than 3m away. We use two evaluation metrics, the absolute coverage area in m 2 (Cov) and percentage of area explored in the scene (% Cov), i.e. ratio of coverage to maximum possible coverage in the corresponding scene. During training, each episode lasts for a fixed length of 1000 steps. We use train/val/test splits provided by Savva et al. 2019 for both the datasets. Note that the set of scenes used in each split is disjoint, which means the agent is tested on new scenes never seen during training. Gibson test set is not public but rather held out on an online evaluation server for the Pointgoal task. We use the validation as the test set for comparison and analysis for the Gibson domain. We do not use the validation set for hyper-parameter tuning. To analyze the performance of all the models with respect to the size of the scene, we split the Gibson validation set into two parts, a small set of 10 scenes with explorable area ranging from 16m 2 to 36m 2, and a large set of 4 scenes with traversable area ranging from 55m 2 to 100m 2. Note that the size of the map is usually much larger than the traversable area, with the largest map being about 23m long and 11m wide. Training Details. We train our model for the Exploration task in the Gibson domain and transfer it to the Matterport domain. The Projection Unit is trained to predict egocentric projections, and the Pose Estimator is trained to predict agent pose using supervised learning. The ground truth egocentric projection is computed using geometric projections from ground truth depth. The Global and Local policies are both trained using Reinforcement Learning. The reward for the Global policy is the increase in coverage and the reward for the Local policy is the reduction in Euclidean distance to the short-term goal. All the modules are trained simultaneously. Their parameters are independent, but the data distribution is inter-dependent. Based on the actions taken by the Local policy, the future input to Mapper changes, which in turn changes the map input to the Global policy and consequently affects the short-term goal given to the Local Policy. For more architecture and hyperparameter details please refer to the supplementary material. We will also open-source the code. Baselines. We use a range of end-to-end Reinforcement Learning (RL) methods as baselines: RL + 3LConv: An RL Policy with 3 layer convolutional network followed by a GRU as described by Savva et al. 2019 which is also identical to our Local Policy architecture. RL + Res18: A RL Policy initialized with ResNet18 pre-trained on ImageNet followed by a GRU. RL + Res18 + AuxDepth: This baseline is adapted from Mirowski et al. 2017 who use depth prediction as an auxiliary task. We use the same architecture as our Mapper (conv layers from ResNet18) with one additional deconvolutional layer for Depth prediction followed by 3 layer convolution and GRU for the policy. RL + Res18 + ProjDepth: This baseline is adapted form who project the depth image in an egocentric top-down in addition to the RGB image as input to the RL policy. Since we do not have depth as input, we use the architecture from RL + Res18 + AuxDepth for depth prediction and project the predicted depth before passing to 3Layer Conv and GRU policy. For all the baselines, we also feed a 32-dimensional embedding of the sensor pose reading to the GRU along with the image-based representation. This embedding is also learnt end-to-end using RL. All baselines are trained using PPO with increase in coverage as the reward. We train the proposed ANM model and all the baselines for the Exploration task with 10 million frames on the Gibson training set. The are shown in Table 1. The on the Gibson Val set are averaged over a total of 994 episodes in 14 different unseen scenes. The proposed model achieves an average absolute and relative coverage of 31.379m 2 /0.924 as compared to 24.958m 2 /0.766 for the best baseline. This indicates that the proposed model is more efficient and effective at exhaustive exploration as compared to the baselines. This is because our hierarchical policy architecture reduces the horizon of the long-term exploration problem as instead of taking tens of low-level navigational actions, the Global policy only takes few long-term goal actions. We also report the domain generalization performance on the Exploration task in Table 1 (see shaded region), where all models trained on Gibson are evaluated on the Matterport domain. ANM leads to higher domain generalization performance (57.228m 2 /0.405 vs 41.549m 2 /0.297). The abosulte coverage is higher' Episode length Gibson Val -Overall for the Matterport domain as it consists of larger scenes on average. Some visualizations of policy executions are provided in the Appendix. In Fig. 3, we plot the relative coverage (% Cov) of all the models as the episode progresses on the large and small scene sets, as well as the overall Gibson Val set. The plot on the small scene set shows that ANM is able to almost completely explore the small scenes in around 500 steps, however the baselines are only able to explore 85% of the small scenes in 1000 steps (see Fig. 3 center). This indicates that ANM explores more efficiently in small scenes. The plot on the large scenes set shows that the performance gap between ANM and baselines widens as the episode progresses (see Fig. 3 left). Looking at the behaviour of the baselines, we saw that they often got stuck in local areas. This behaviour indicates that they are unable to remember explored areas over long-time horizons and are ineffective at long-term planning. On the other hand, ANM uses a Global policy on the map which allows it to have memory of explored areas over long-time horizons, and plan effectively to reach distant long-term goals by leveraging analytical planners. As a , it is able to explore effectively in large scenes with long episode lengths. Local Policy. An alternative to learning a Local Policy is to have a deterministic policy which follows the plan given by the Planner. As shown in Table 2, the ANM model performs much worse without the Local Policy. The Local Policy is designed to adapt to small errors in Mapping. We observed Local policy overcoming both false positives and false negatives encountered in mapping. For example, the Mapper could sometime wrongly predict a carpet as an obstacle. In this case, the planner would plan to go around the carpet. However, if the short-term goal is beyond the carpet, the Local policy can understand that the carpet is not an obstacle based on the RGB observation and learn to walk over it. Similarly, we also observed cases where the Mapper didn't predict small obstacles very close to the agent as they were not in the field-of-view due to the height of the camera. In this case, the planner would plan a path through the obstacle where the deterministic policy would get stuck. Since the local policy is recurrent, it learns to navigate around these obstacles by getting feedback from the environment. When the policy tries to move forward but it can not, it gets feedback that there must be an obstacle. Global Policy. An alternative to learning a Global Policy for sampling long-term goals is to use a classical algorithm called Frontier-based exploration . A frontier is defined as the boundary between the explored free space and the unexplored space. Frontier-based exploration essentially sample points on this frontier as goals to explore the space. There are different variants of Frontier-based exploration based on the sampling strategy. compare different sampling strategies and find that sampling the point on the frontier closest to the agent gives the best empirically. We implement this variant and replace it with our learned Global Policy. As shown in Table 2, Frontier-based exploration policy perform worse than the Global Policy. We observed that Frontier-based exploration spent a lot of time exploring corners or small area behind furniture. In contrast, the trained Global policy ignored small spaces and chose distant long-term goals which led to exploring more area. Pose Estimation. A difference between ANM and the baselines is that ANM uses additional supervision to train the Pose Estimator. In order to understand whether the performance gain is coming from this additional supervision, we remove the Pose Estimator from ANM and just use the input sensor reading as our pose estimate. Results in Table 2 shows that the ANM still outperforms the baselines even without the Pose Estimator. We also tried passing the ground truth pose as input the baselines instead of the sensor reading. The performance of the baselines did not improve with the ground truth pose. We deploy the trained ANM policy on a Locobot in the real-world. In order to match the real-world observations to the simulator observations as closely as possible, we change the simulator input configuration to match the camera intrinsics on the Locobot. This includes the camera height and horizontal and vertical field-of-views. In Figure 4, we show an episode of ANM exploring the living area in an apartment. The figure shows that the policy transfers well to the real-world and is able to effectively explore the environment. The long-term goals sampled by the Global policy (shown by blue circles on the map) are often towards open spaces in the explored map, which indicates that it is learning to exploit the structure in the map. the goal location. In addition to Success rate (Succ), Success weighted by (normalized inverse) Path Length or SPL is also used as a metric for evaluation as proposed by Anderson et al. 2018. All the baseline models trained for the task of Exploration either need to be retrained or atleast fine-tuned to be transferred to the Pointgoal task. The modularity of ANM provides it another advantage that it can be transferred to the Pointgoal task without any additional training. For transfer to the Pointgoal task, we just fix the Global policy to always output the PointGoal coordinates as the long-term goal and use the Local and Mapper trained for the Exploration task. We found that an ANM policy trained on exploration, when transferred to the Pointgoal task performed better than several RL and Imitation Learning baselines trained on the Pointgoal task. The transferred ANM model achieves a success rate/SPL of 0.950/0.846 as compared to 0.827/0.730 for the best baseline model on Gibson val set. The ANM model also generalized significantly better than the baselines to harder goals and to the Matterport domain. In addition to better performance, ANM was also 10 to 75 times more sample efficient than the baselines. This transferred ANM policy was also the winner of the CVPR 2019 Habitat Pointgoal Navigation Challenge for both RGB and RGB-D tracks among over 150 submissions from 16 teams. These highlight a key advantage of our model that it allows us to transfer the knowledge of obstacle avoidance and control in low-level navigation across tasks, as the Local Policy and Mapper are task-invariant. More details about the Pointgoal experiments, baselines, including domain and goal generalization on the Pointgoal task are provided in the supplementary material. In this paper, we proposed a modular navigational model which leverages the strengths of classical and learning-based navigational methods. We show that the proposed model outperforms prior methods on both Exploration and PointGoal tasks and shows strong generalization across domains, goals, and tasks. In future, the proposed model can be extended to complex semantic tasks such as Semantic Goal Navigation and Embodied Question Answering by using a semantic Mapper which creates multi-channel map capturing semantic properties of the objects in the environment. The model can also be combined with prior work on Localization to relocalize in a previously created map for efficient navigation in subsequent episodes. PointGoal has been the most studied task in recent literature on navigation where the objective is to navigate to a goal location whose relative coordinates are given as input in a limited time budget. We follow the PointGoal task setup from Savva et al. 2019, using train/val/test splits for both Gibson and Matterport datasets. Note that the set of scenes used in each split is disjoint, which means the agent is tested on new scenes never seen during training. Gibson test set is not public but rather held out on an online evaluation server 5. We report the performance of our model on the Gibson test set when submitted to the online server but also use the validation set as another test set for extensive comparison and analysis. We do not use the validation set for hyper-parameter tuning. Savva et al. 2019 identify two measures to quantify the difficulty of a PointGoal dataset. The first is the average geodesic distance (distance along the shortest path) to the goal location from the starting location of the agent, and the second is the average geodesic to Euclidean distance ratio (GED ratio). The GED ratio is always greater than or equal to 1, with higher ratio ing in harder episodes. The train/val/test splits in Gibson dataset come from the same distribution of having similar average geodesic distance and GED ratio. In order to analyze the performance of the proposed model on out-of-set goal distribution, we create two harder sets, Hard-Dist and Hard-GEDR. In the Hard-Dist set, the geodesic distance to goal is always more than 10m and the average geodesic distance to the goal is 13.48m as compared to 6.9/6.5/7.0m in train/val/test splits . Hard-GEDR set consists of episodes with an average GED ratio of 2.52 and a minimum GED ratio of 2.0 as compared to average GED ratio 1.37 in the Gibson val set. We also follow the episode specification from Savva et al. 2019. Each episode ends when either the agent takes the stop action or at a maximum of 500 timesteps. An episode is considered a success when the final position of the agent is within 0.2m of the goal location. In addition to Success rate (Succ), we also use Success weighted by (normalized inverse) Path Length or SPL as a metric for evaluation for the PointGoal task as proposed by Anderson et al. 2018. In Table 3, we show the performance of the proposed model transferred to the PointGoal task along with the baselines trained on the PointGoal task with the same amount of data (10million frames). The proposed model achieves a success rate/SPL of 0.950/0.846 as compared to 0.827/0.730 for the best baseline model on Gibson val set. We also report the performance of the proposed model trained from scratch on the PointGoal task for 10 million frames. The indicate that the performance of ANM transferred from Exploration is comparable to ANM trained on PointGoal. This highlights a key advantage of our model that it allows us to transfer the knowledge of obstacle avoidance and control in low-level navigation across tasks, as the Local Policy and Mapper are task-invariant. On the left, we show some successful trajectories which indicate that the model is effective at long distance goals with high GED ratio. On the right, we show a failure case due to mapping error. Sample efficiency. RL models are typically trained for more than 10 million samples. In order to compare the performance and sample-efficiency, we trained the best performing RL model (RL + Res18 + GRU + ProjDepth) for 75 million frames and it achieved a Succ/SPL of 0.678/0.486. ANM reaches the performance of 0.789/0.703 SPL/Succ at only 1 million frames. These numbers indicate that ANM achieves > 75× speedup as compared to the best RL baseline. Table 3 (see shaded region), we evaluate all the baselines and ANM trained on the PointGoal task in the Gibson domain on the test set in Matterport domain as well as the harder goal sets in Gibson. We also transfer ANM trained on Exploration in Gibson on all the 3 sets. The show that ANM outperforms all the baselines at all generalization sets. Interestingly, RL based methods almost fail completely on the Hard-Dist set. We also analyze the performance of the proposed model as compared to two best baselines CMP and IL + Res18 + GRU as a function of geodesic distance to goal and GED ratio in Figure 6. The performance of the baselines drops faster as compared to ANM, especially with increase in goal distance. This indicates that end-toend learning methods are effective at short-term navigation but struggle when long-term planning is required to reach a distant goal. In Figure 9, we show some example trajectories of the ANM model along with the predicted map. The successful trajectories indicate that the model exhibits strong backtracking behavior which makes it effective at distant goals requiring long-term planning. In order to implement the actuation and sensor noise models, we would like to collect data for navigational actions in the Habitat simulator. We use three default navigational actions: Forward: move forward by 25cm, Turn Right: on the spot rotation clockwise by 10 degrees, and Turn Left: on the spot rotation counter-clockwise by 10 degrees. The control commands are implemented as u F orward = (0.25, 0, 0), u Right: (0, 0, −10 * π/180) and u Lef t: (0, 0, 10 * π/180). In practice, a robot can also rotate slightly while moving forward and translate a bit while rotating on-the-spot, creating rotational actuation noise in forward action and similarly, a translation actuation noise in on-the-spot rotation actions. We use a Locobot 7 to collect data for building the actuation and sensor noise models. We use the pyrobot API along with ROS to implement the control commands and get sensor readings. In order to get an accurate agent pose, we use an Hokuyo UST-10LX Scanning Laser Rangefinder (LiDAR) which is especially very precise in our scenario as we take static readings in 2D . We install the LiDAR on the Locobot by replacing the arm with the LiDAR. We note that the Hokuyo UST-10LX Scanning Laser Rangefinder is an expensive sensor. It costs $1600 as compared to the whole Locobot costing less than $2000 without the arm. Using expensive sensors can improve the performance of a model, however for a method to be scalable, it should ideally work with cheaper sensors too. In order demonstrate the scalability of our method, we use the LiDAR only to collect the data for building noise models and not for training or deploying navigation policies in the real-world. For the sensor estimate, we use the Kobuki base odometry available in Locobot. We approximate the LiDAR pose estimate to be the true pose of the agent as it is orders of magnitude more accurate than the base sensor. For each action, we collect 600 datapoints from both the base sensor and the LiDAR, making a total of 3600 datapoints (600 * 3 * 2). We use 500 datapoints for each action to fit the actuation and sensor noise models and use the remaining 100 datapoints for validation. For each action a, the LiDAR pose estimates gives us samples of p 1 and the base sensor readings give us samples of p 1, i = 1, 2,..., 600. The difference between LiDAR estimates (p For each action a, we fit a separate Gaussian Mixture Model for the actuation noise and sensor noise using samples i act,a and i sen,a respectively, making a total of 6 models. We fit Gaussian mixture models with number of components ranging from 1 to 20 for and pick the model with highest likelihood on the validation set. Each component in these Gaussian mixture models is a multi-variate Gaussian in 3 variables, x, y and o. We implement these actuation and sensor noise models in the Habitat simulator for our experiments. The Mapper (f M ap) takes in the current RGB observation, s t ∈ R 3×H×W, the current and last sensor reading of the agent pose x t−1:t and the map at the previous time step m t−1 ∈ R 2×M ×M and outputs an updated map, m t ∈ R 2×M ×M (see Figure 2): where θ M denote the trainable parameters and p t−1 denotes internal representations of the Mapper. The Mapper can be broken down into two parts, a Projection Unit (f P r) and a Pose Estimator Unit (f P E,). The Projection Unit outputs a egocentric top-down 2D spatial map, p ego t (where V is the vision range), predicting the obstacles and the explored area in the current observation: p ego t = f P r (s t |θ P r), where θ P r are the parameters of the Projection Unit. It consists of Resnet18 convolutional layers to produce an embedding of the observation. This embedding is passed through two fully-connected layers followed by 3 deconvolutional layers to get the first-person top-down 2D spatial map prediction. Now, we would like to add the egocentric map prediction (p ego t) to the geocentric map from the previous time step (m t−1). In order to transform the egocentric map to the geocentric frame, we need the pose of the agent in the geocentric frame. The sensor reading x t is typically noisy. Thus, we have a Pose Estimator to correct the sensor reading and give an accurate estimate of the agent's geocentric pose. In order to estimate the pose of the agent, we first calculate the relative pose change (dx) from the last time step using the sensor readings at the current and last time step (x t−1, x t). Then we use a Spatial Transformation on the egocentric map prediction at the last frame (p ego t−1) based on the relative pose change (dx), p t−1 = f ST (p ego t−1 |dx). Note that the parameters of this Spatial Transformation are not learnt, but calculated using the pose change (dx). This transforms the projection at the last step to the current egocentric frame of reference. If the sensor was accurate, p t−1 would highly overlap with p ego t. The Pose Estimator Unit takes in p t−1 and p ego t as input and predicts the relative pose change:d x t = f P E (p t−1, p ego t |θ P E) The intuition is that by looking at the egocentric predictions of the last two frames, the pose estimator can learn to predict the small translation and/or rotation that would align them better. The predicted relative pose change is then added to the last pose estimate to get the final pose estimatex t =x t−1 +d x t. Finally, the egocentric spatial map prediction is transformed to the geocentric frame using the current pose prediction of the agent (x t) using another Spatial Transformation and aggregated with the previous spatial map (m t−1) using Channel-wise Pooling operation: m t = m t−1 + f ST (p t |x t). m t = f M ap (s t, x t−1:t, m t−1 |θ M, b t−1) = m t−1 + f ST (p t |x t + f P E (f ST (p ego t−1 |x t−1:t), f P r (s t |θ P r)|θ P E )) where θ P r, θ P E ∈ θ M, and p We use for implementing and training our model. The Projection Unit in the mapper consists of ResNet18 convolutional layers followed by 2 fully-connected layers trained with dropout of 0.5, followed by 3 deconvolutional layers. The Pose Estimator consists of 3 convolutional layers followed by 2 fully connected layers. The Global Policy is a 5 layer fully-convolutional network, while the Local Policy consists of a 3-layer Convolutional network followed by a GRU. The Global and Local policies are both trained using Reinforcement Learning. The reward for the Global policy is the increase in coverage and the reward for the Local policy is the reduction in Euclidean distance to the short-term goal. implementation of Global and Local policy is based on Kostrikov 2018. In addition to the RGB observation, the Local policy receives relative distance and angle to short-term goal, current timestep and last action as input. We bin the relative distance (bin size increasing with distance), relative angle (5 degree bins) and current timestep (30 time step bins) before passing them through embedding layers. This kind of discretization is used previously for RL policies and it improved the sample efficiency as compared to passing the continuous values as input directly. For fair comparison, we use the same discretization for all the baselines as well. We train all the components with 72 parallel threads, with each thread using one of the 72 scenes in the Gibson training set. This leads to a batch size of 72 for training the Mapper. The Global policy samples a new goal every 25 timesteps. We use for training the Global and Local policies with 72 parallel threads and a horizon length of 25 steps for the Local policy and 20 steps for the Global policy (20 steps for Global policy is equivalent to 500 low-level timesteps as Global policy samples a new goal after every 25 timesteps). We use Adam optimizer with a learning rate of 0.0001 for training both the units in the Mapper and Adam with a learning rate of 0.00025 for training the Global and Local policies. We use a discount factor of γ = 0.99, entropy coefficient of 0.001, value loss coefficient of 0.5 for training both the Global and Local policies. Input frame size is 128 × 128, the vision range for mapper is V = 64, i.e. 3.2m (each cell is 5cm in length). Since there are no parameters dependent on the map size, it can be adaptive. We train with a map size of M = 960 (equivalent to 48m). A map of size 48m × 48m is large enough for all scenes in the Gibson val set. We use an adaptive map size for Pointgoal evaluation such that goal lies within central 50% of the map to handle even larger maps in the unseen test set. For the exploration task, we train and test with a constant M = 960. For the Global policy in the Exploration task, the size of the Global Policy input is G = 240.

A modular and hierarchical approach to learn policies for exploring 3D environments.

We propose an end-to-end differentiable planning network for graphs. This can be applicable to many motion planning problems


Question: Represent the article for finding a single-sentence TLDR

Question: Deep Learning for Computer Vision depends mainly on the source of supervision. Photo-realistic simulators can generate large-scale automatically labeled synthetic data, but introduce a domain gap negatively impacting performance. We propose a new unsupervised domain adaptation algorithm, called SPIGAN, relying on Simulator Privileged Information (PI) and Generative Adversarial Networks (GAN). We use internal data from the simulator as PI during the training of a target task network. We experimentally evaluate our approach on semantic segmentation. We train the networks on real-world Cityscapes and Vistas datasets, using only unlabeled real-world images and synthetic labeled data with z-buffer (depth) PI from the SYNTHIA dataset. Our method improves over no adaptation and state-of-the-art unsupervised domain adaptation techniques. Learning from as little human supervision as possible is a major challenge in Machine Learning. In Computer Vision, labeling images and videos is the main bottleneck towards achieving large scale learning and generalization. Recently, training in simulation has shown continuous improvements in several tasks, such as optical flow BID32, object detection BID31 BID52 BID47 BID36, tracking BID10, pose and viewpoint estimation BID44 BID34 BID46, action recognition BID9, and semantic segmentation BID15 BID39 BID38. However, large domain gaps between synthetic and real domains remain as the main handicap of this type of strategies. This is often addressed by manually labeling some amount of real-world target data to train the model on mixed synthetic and real-world labeled data (supervised domain adaptation). In contrast, several recent unsupervised domain adaptation algorithms have leveraged the potential of Generative Adversarial Networks (GANs) BID14 for pixel-level adaptation in this context BID1 BID45. These methods often use simulators as black-box generators of (x, y) input / output training samples for the desired task. Our main observation is that simulators internally know a lot more about the world and how the scene is formed, which we call Privileged Information (PI). This Privileged Information includes physical properties that might be useful for learning. This additional information z is not available in the real-world and is, therefore, generally ignored during learning. In this paper, we propose a novel adversarial learning algorithm, called SPIGAN, to leverage Simulator PI for GAN-based unsupervised learning of a target task network from unpaired unlabeled real-world data. We jointly learn four different networks: (i) a generator G (to adapt the pixel-level distribution of synthetic images to be more like real ones), (ii) a discriminator D (to distinguish adapted and real images), (iii) a task network T (to predict the desired label y from image x), and (iv) a privileged network P trained on both synthetic images x and adapted ones G(x) to predict their associated privileged information z. Our main contribution is a new method to leverage PI from a simulator via the privileged network P, which acts as an auxiliary task and regularizer to the task network T, the main output of our SPIGAN learning algorithm. We evaluate our approach on semantic segmentation in urban scenes, a challenging real-world task. We use the standard Cityscapes BID6 and Vistas BID33 datasets as target real-world data (without using any of the training labels) and SYNTHIA BID39 as simulator output. Although our method applies to any kind of PI that can be predicted via a deep network (optical flow, instance segmentation, object detection, material properties, forces, ...), we consider one of the most common and simple forms of PI available in any simulator: depth from its z-buffer. We show that SPIGAN can successfully learn a semantic segmentation network T using no real-world labels, partially bridging the sim-to-real gap (see Figure 1). SPIGAN also outperforms related state-of-the-art unsupervised domain adaptation methods. The rest of the paper is organized as follows. Section 2 presents a brief review of related works. Section 3 presents our SPIGAN unsupervised domain adaptation algorithm using simulator privileged information. We report our quantitative experiments on semantic segmentation in Section 4, and conclude in Section 5. Domain adaptation (cf. BID7 for a recent review) is generally approached either as domaininvariant learning BID19 BID17 BID11 or as a statistical alignment problem BID50 BID28. Our work focuses on unsupervised adaptation methods in the context of deep learning. This problem consists in learning a model for a task in a target domain (e.g., semantic segmentation of real-world urban scenes) by combining unlabeled data from this domain with labeled data from a related but different source domain (e.g., synthetic data from simulation). The main challenge is overcoming the domain gap, i.e. the differences between the source and target distributions, without any supervision from the target domain. The Domain Adversarial Neural Network (DANN) BID50 BID11 BID12 ) is a popular approach that learns domain invariant features by maximizing domain confusion. This approach has been successfully adopted and extended by many other researchers, e.g., BID37;. Curriculum Domain Adaptation ) is a recent evolution for semantic segmentation that reduces the domain gap via a curriculum learning approach (solving simple tasks first, such as global label distribution in the target domain).Recently, adversarial domain adaptation based on GANs BID14 have shown encouraging for unsupervised domain adaptation directly at the pixel level. These techniques learn a generative model for source-to-target image translation, including from and to multiple domains BID48 BID45 BID24. In particular, CycleGAN leverages cycle consistency using a forward GAN and a backward GAN to improve the training stability and performance of image-to-image translation. An alternative to GAN is Variational Auto-Encoders (VAEs), which have also been used for image translation.Several related works propose GAN-based unsupervised domain adaptation methods to address the specific domain gap between synthetic and real-world images. SimGAN BID45 leverages simulation for the automatic generation of large annotated datasets with the goal of refining synthetic images to make them look more realistic. Sadat BID40 effectively leverages synthetic data by treating foreground and in different manners. Similar to our approach, xs (synthetic) ys (label) zs (privileged info.) DISPLAYFORM0 Figure 2: SPIGAN learning algorithm from unlabeled real-world images x r and the unpaired output of a simulator (synthetic images x s, their labels y s, e.g. semantic segmentation ground truth, and Privileged Information PI z s, e.g., depth from the z-buffer) modeled as random variables. Four networks are learned jointly: (i) a generator G(x s) ∼ x r, (ii) a discriminator D between G(x s) = x f and x r, (iii) a perception task network T (x r) ∼ y r, which is the main target output of SPIGAN (e.g., a semantic segmentation deep net), and (iv) a privileged network P to support the learning of T by predicting the simulator's PI z s.recent methods consider the final recognition task during the image translation process. Closely related to our work, PixelDA BID1 is a pixel-level domain adaptation method that jointly trains a task classifier along with a GAN using simulation as its source domain but no privileged information. These approaches focus on simple tasks and visual conditions that are easy to simulate, hence having a low domain gap to begin with. On the other hand, BID21 are the first to study semantic segmentation as the task network in adversarial training. uses a curriculum learning style approach to reduce domain gap. BID41 conducts domain adaptation by utilizing the task-specific decision boundaries with classifiers. BID42 leverage the GAN framework by learning general representation shared between the generator and segmentation networks. BID5 use a target guided distillation to encourage the task network to imitate a pretrained model. BID57 propose to combine appearance and representation adaptation. BID49 propose an adversarial learning method to adapt in the output (segmentation) space. BID59 generates pseudo-labels based on confidence scores with balanced class distribution and propose an iterative self-training framework. Our main novelty is the use of Privileged Information from a simulator in a generic way by considering a privileged network in our architecture (see Figure 2). We show that for the challenging task of semantic segmentation of urban scenes, our approach significantly improves by augmenting the learning objective with our auxiliary privileged task, especially in the presence of a large sim-to-real domain gap, the main problem in challenging real-world conditions. Our work is inspired by Learning Using Privileged Information (LUPI) BID51, which is linked to distillation BID18 as shown by BID29. LUPI's goal is to leverage additional data only available at training time. For unsupervised domain adaptation from a simulator, there is a lot of potentially useful information about the generation process that could inform the adaptation. However, that information is only available at training time, as we do not have access to the internals of the real-world data generator. Several works have used privileged information at training time for domain adaptation BID2 BID20 BID2 BID43 BID13. BID20 leverage RGBD information to help adapt an object detector at the feature level, while BID13 propose a similar concept of modality distillation for action recognition. Inspired by this line of work, we exploit the privileged information from simulators for sim-to-real unsupervised domain adaptation. Our goal is to design a procedure to learn a model (neural network) that solves a perception task (e.g., semantic segmentation) using raw sensory data coming from a target domain (e.g., videos of a car driving in urban environments) without using any ground truth data from the target domain. We formalize this problem as unsupervised domain adaptation from a synthetic domain (source domain) to a real domain (target domain). The source domain consists of labeled synthetic images together with Privileged Information (PI), obtained from the internal data structures of a simulator. The target domain consists of unlabeled images. The simulated source domain serves as an idealized representation of the world, offering full control of the environment (weather conditions, types of scene, sensor configurations, etc.) with automatic generation of raw sensory data and labels for the task of interest. The main challenge we address in this work is how to overcome the gap between this synthetic source domain and the target domain to ensure generalization of the task network in the real-world without target supervision. Our main hypothesis is that the PI provided by the simulator is a rich source of information to guide and constrain the training of the target task network. The PI can be defined as any information internal to the simulator, such as depth, optical flow, or physical properties about scene components used during simulation (e.g., materials, forces, etc.). We leverage the simulator's PI within a GAN framework, called SPIGAN. Our approach is described in the next section. DISPLAYFORM0 s ), i = 1... N s } be a set of N s simulated images x s with their labels y s and PI z s. We describe our approach assuming a unified treatment of the PI, but our method trivially extends to multiple separate types of PI. DISPLAYFORM1, and (iv) a privileged network P (x; θ P). The generator G is a mapping function, transforming an image x s in X s (source domain) to x f in X f (adapted or fake domain). SPIGAN aims to make the adapted domain statistically close to the target domain to maximize the accuracy of the task predictor T (x; θ T) during testing. The discriminator D is expected to tell the difference between x f and x r, playing an adversarial game with the generator until a termination criteria is met (refer to section 4.1). The target task network T is learned on the synthetic x s and adapted G(x s ; θ G) images to predict the synthetic label y s, assuming the generator presents a reasonable degree of label (content) preservation. This assumption is met for the regime of our experiments. Similarly, the privileged network P is trained on the same input but to predict the PI z, which in turn assumes the generator G is also PI-preserving. During testing only T (x; θ T) is needed to do inference for the selected perception task. The main learning goal is to train a model θ T that can correctly perform a perception task T in the target real-world domain. All models are trained jointly in order to exploit all available information to constrain the solution space. In this way, the PI provided by the privileged network P is used to constrain the learning of T and to encourage the generator to model the target domain while being label-and PI-preserving. Our joint learning objective is described in the following section. We design a consistent set of loss functions and domain-specific constraints related to the main prediction task T. We optimize the following minimax objective: min DISPLAYFORM0 where α, β, γ, δ are the weights for adversarial loss, task prediction loss, PI regularization, and perceptual regularization respectively, further described below. Adversarial loss L GAN. Instead of using a standard adversarial loss, we use a least-squares based adversarial loss BID30;, which stabilizes the training process and generates better image in our experiments: DISPLAYFORM1 where P r (resp. P s) denotes the real-world (resp. synthetic) data distribution. Task prediction loss L T. We learn the task network by optimizing its loss over both synthetic images x s and their adapted version G(x s, θ G). This assumes the generator is label-preserving, i.e., that y s can be used as a label for both images. Thanks to our joint objective, this assumption is directly encouraged during the learning of the generator through the joint estimation of θ P, which relates to scene properties captured by the PI. Naturally, different tasks require different loss functions. In our experiments, we consider the task of semantic segmentation and use the standard cross-entropy loss (Eq. 4) over images of size W × H and a probability distribution over C semantic categories. The total combined loss in the special case of semantic segmentation is therefore: DISPLAYFORM2 DISPLAYFORM3 where 1 [a=b] is the indicator function. PI regularization L P. Similarly, the auxiliary task of predicting PI also requires different losses depending on the type of PI. In our experiments, we use depth from the z-buffer and an 1 -norm: DISPLAYFORM4 Perceptual regularization L perc. To maintain the semantics of the source images in the generated images, we additionally use the perceptual loss BID23; BID3: DISPLAYFORM5 where φ is a mapping from image space to a pre-determined feature space (see 4.1 for more details).Optimization. In practice, we follow the standard adversarial training strategy to optimize our joint learning objective (Eq. 1). We alternate between updates to the parameters of the discriminator θ D, keeping all other parameters fixed, then fix θ D and optimize the parameters of the generator θ G, the privileged network θ P, and most importantly the task network θ T. We discuss the details of our implementation, including hyper-parameters, in section 4.1. We evaluate our unsupervised domain adaptation method on the task of semantic segmentation in a challenging real-world domain for which training labels are not available. As our source synthetic domain, we select the public SYNTHIA dataset BID39 as synthetic source domain given the availability of automatic annotations and PI. SYNTHIA is a dataset generated from an autonomous driving simulator of urban scenes. These images were generated under different weathers and illumination conditions to maximize visual variability. Pixel-wise segmentation and depth labels are provided for each image. In our experiment, we use the sequence of SYNTHIA-RAND-CITYSCAPES, which contains semantic segmentation labels that are more compatible with Cityscapes. For target real-world domains, we use the Cityscapes BID6 and Mapillary Vistas BID33 datasets. Cityscapes is one of most widely used real-world urban scene image segmentation datasets with images collected around urban streets in Europe. For this dataset, We use the standard split for training and validation with 2, 975 and 500 images respectively. Mapillary Vistas is a larger dataset with a wider variety of scenes, cameras, locations, weathers, and illumination conditions. We use 16, 000 images for training and 2, 000 images for evaluation. During training, none of the labels from the real-world domains are used. In our experiment, we first evaluate adaptation from SYNTHIA to Cityscapes on 16 classes, following the standard evaluation protocol used in Hoffman et al. of using PI by conducting ablation study with and without PI (depth) during adaptation from SYN-THIA to both Cityscapes and Vistas, on a common 7 categories ontology. To be consistent with the semantic segmentation best practices, we use standard intersection-over-union (IoU) per category and mean intersection-over-union (mIoU) as our main validation metric. We adapt the generator and discriminator model architectures from CycleGAN and BID23. For simplicity, we use a single sim-to-real generator (no cycle consistency) consisting of two down-sampling convolution layers, nine ResNet blocks BID16 and two fractionally-strided convolution layers. Our discriminator is a PatchGAN network with 3 layers. We use the standard FCN8s architecture BID28 for both the task predictor T and the privileged network P, given its ease of training and its acceptance in domain adaptation works BID21. For the perceptual loss L perc, we follow the implementation in BID3. The feature is constructed by the concatenation of the activations of a pre-trained VGG19 network BID53 of layers "conv1 2", "conv2 2", "conv3 2", "conv4 2", "conv5 2". FORMULA3, we set hyper-parameters using a coarse grid search on a small validation set different than the target set. For Cityscapes, we use a subset of the validation set of Vistas, and vice-versa. We found a set of values that are effective across datasets and experiments, which show they have a certain degree of robustness and generalization. The weights in our joint adversarial loss (Eq. 1) are set to α = 1, β = 0.5, γ = 0.1, δ = 0.33, for the GAN, task, privileged, and perceptual objectives respectively. This confirms that the two most important factors in the objective are the GAN and task losses (α = 1, β = 0.5). This is intuitive, as the goal is to improve the generalization performance of the task network (the task loss being an empirical proxy) across a potentially large domain gap (addressed first and foremost by the GAN loss). The regularization terms are secondary in the objective, stabilizing the training (perceptual loss) and constraining the adaptation process (privileged loss). FIG1 show an example of our loss curves and the stability of our training. Another critical hyper-parameter for unsupervised learning is the stopping criterion. We observed that the stabilizing effects of the task and privileged losses (Eqs. 3,5) on the GAN objective (Eq. 2) made a simple rule effective for early stopping. We stop training at the iteration when the discriminator loss is significantly and consistently better than the generator loss (iteration 90 in FIG2). This is inspired by the semi-supervised of BID8, where effective discriminative adaptation of the task network might not always be linked to the best image generator. We evaluate the methods with two resolutions: 320 × 640 and 512 × 1024, respectively. Images are resized to the evaluated size during training and evaluation. During training, we sample crops of size 320 × 320 (resp. 400 × 400) for lower (resp. higher) resolution experiments. In all adversarial learning cases, we do five steps of the generator for every step of the other networks. The Adam optimizer BID25 ) is used to adjust all parameters with initial learning rate 0.0002 in our PyTorch implementation BID35. n/a n/a n/a n/a n/a n/a n/a n/a n/a n/a n/a n/a n/a n/a n/a n/a 23.2 LSD n/a n/a n/a n/a n/a n/a n/a n/a n/a n/a n/a n/a n/a n/a n/a n/a 34. Table 2: Semantic Segmentation (per category and mean IoUs, higher is better) for SYN-THIA adapting to Cityscapes and Vistas. The last column is the ratio of images in the validation set for which we observe negative transfer (lower is better). In this section we present our evaluation of the SPIGAN algorithm in the context of adapting a semantic segmentation network from SYNTHIA to Cityscapes. Depth maps from SYNTHIA are used as PI in the proposed algorithm. We compare our to several state-of-art domain adaptation algorithms, including FCNs in the wild (FCNs wild) BID21, Curriculum DA (CDA), Learning from synthetic data (LSD) BID42, and Class-balanced Self-Training (CBST) BID59.Quantitative for these methods are shown in Table 1 for the semantic segmentation task on the target domain of Cityscapes (validation set). As reference baselines, we include training only on source images and non-adapted labels. We also provide our algorithm performance without the PI for comparison (i.e., γ = 0 in Eq. 1, named "SPIGAN-no-PI").Results show that on Cityscapes SPIGAN achieves state-of-the-art semantic segmentation adaptation in terms of mean IoU. A finer analysis of the attending to individual classes suggests that the use of PI helps to estimate layout-related classes such as road and sidewalk and object-related classes such as person, rider, car, bus and motorcycle. SPIGAN achieves an improvement of 3% in 320 × 640, 1.0% in 512 × 1024, in mean IoU with respect to the non-PI method. This improvement is thanks to the regularization provided by P (x; θ P) during training, which decreases the number of artifacts as shown in Figure 5. This comparison, therefore, confirms our main contribution: a general approach to leveraging synthetic data and PI from the simulator to improve generalization performance across the sim-to-real domain gap. To better understand the proposed algorithm, and the impact of PI, we conduct further experiments comparing SPIGAN (with PI), SPIGAN-no-PI (without PI), and SPIGAN-base (without both PI and perceptual regularization), the task network of SPIGAN trained only on the source domain (FCN source, lower bound, no adaptation), and on the target domain (FCN target, upper bound), all at 320 × 640 resolution. We also include on the Vistas dataset, which presents a more challenging adaptation problem due to the higher diversity of its images. For these experiments, we use a 7 semantic classes ontology to produce a balanced ontology common to the three datasets (SYNTHIA, Cityscapes and Vistas). Adaptation for both target domains are given in Table 2.In addition to the conventional segmentation performance metrics, we also carried out a study to measure the amount of negative transfer, summarized in Table 2. A negative transfer case is defined as a real-world testing sample that has a mIoU lower than the FCN source prediction (no adaptation).As shown in Table 2, SPIGAN-no-PI, including perceptual regularization, performs better than SPIGAN-base in both datasets. The performance is generally improved in all categories, which implies that perceptual regularization effectively stabilizes the adaptation during training. For Cityscapes, the quantitative in Table 2 show that SPIGAN is able to provide dramatic adaptation as hypothesized. SPIGAN improves the mean IoU by 17.1%, with the PI itself providing an improvement of 7.4%. This is consistent with our observation in the previous experiment (Table 1). We also notice that SPIGAN gets significant improvements on "nature", "construction", and "vehicle" categories. In addition, SPIGAN is able to improve the IoU by +15% on the "human" category, a difficult class in semantic segmentation. We provide examples of qualitative for the adaptation from SYNTHIA to Cityscapes in Figure 5 and Figure 7.On the Vistas dataset, SPIGAN is able to decrease the domain gap by +4.3% mean IoU. In this case, using PI is crucial to improve generalization performance. SPIGAN-no-PI indeed suffers from negative transfer, with its adapted network performing −13% worse than the FCN source without adaptation. Table 2 shows that 80% of the evaluation images have a lower individual IoU after adaptation in the SPIGAN-no-PI case (vs. 42% in the SPIGAN case).The main difference between the Cityscapes and Vistas is due to the difference in visual diversity between the datasets. Cityscapes is indeed a more visually uniform benchmark than Vistas: it was recorded in a few German cities in nice weather, whereas Vistas contains crowdsourced data from all over the world with varying cameras, environments, and weathers. This makes Cityscapes more amenable to image translation methods (including SPIGAN-no-PI), as can be seen in Figure 5 where a lot of the visual adaptation happens at the color and texture levels, whereas Figure 6 shows that SYNTHIA images adapted towards Vistas contain a lot more artifacts. Furthermore, a larger domain gap is known to increase the risk of negative transfer (cf. BID7). This is indeed what we quantitatively measured in Table 2 and qualitatively confirmed in Figure 6. SPIGAN suffers from similar but less severe artifacts. As shown in Figure 6, they are more consistent with the depth of the scene, which helps addressing the domain gap and avoids the catastrophic failures visible in the SPIGAN-no-PI case. This consistent improvement brought by PI in both of the experiments not only shows that PI imposes useful constraints that promote better task-oriented training, but also implies that PI more robustly guides the training to reduce domain shift. By comparing the on the two different datasets, we also found that all the unsupervised adaptation methods share some similarity in the performance of certain categories. For instance, the "vehicle" category has seen the largest improvement for both Cityscapes and Vistas. This trend is consistent with the well-known fact that "object" categories are easier to adapt than "stuff" BID52. However, the same improvement did not appear in the "human" category mainly because the SYNTHIA subset we used in our experiments contains very few humans. This phenomenon has been recently studied in Sadat BID40. We present SPIGAN, a novel method for leveraging synthetic data and Privileged Information (PI) available in simulated environments to perform unsupervised domain adaptation of deep networks. Our approach jointly learns a generative pixel-level adaptation network together with a target task network and privileged information models. We showed that our approach is able to address large domain gaps between synthetic data and target real-world domains, including for challenging realworld tasks like semantic segmentation of urban scenes. For future work, we plan to investigate SPIGAN applied to additional tasks, with different types of PI that can be obtained from simulation.

Represent the TLDR for finding the corresponding scientific article:

An unsupervised sim-to-real domain adaptation method for semantic segmentation using privileged information from a simulator with GAN-based image translation.

represent this text to find a much much longer article about it with say hundreds-thousands of words!

We propose the fusion discriminator, a novel architecture for incorporating conditional information into the discriminator of GANs for structured prediction tasks.