High-Precision Localization Using Ground Texture
LINGUANG ZHANG, Princeton University
ADAM FINKELSTEIN, Princeton University
SZYMON RUSINKIEWICZ, Princeton University

Light Shield

Jetson TX1

…

arXiv:1710.10687v1 [cs.CV] 29 Oct 2017

Query Image

LED

Camera

a

Feature Extraction

Database Images

b

c

d

e

f

Fig. 1. System pipeline. (a) Our test robot features an NVIDIA Jetson TX1 development board, which controls a Point Grey mono camera. A light shield
and ring of LEDs around the camera provide controlled lighting. (b) We first capture database images as a preprocess, and (c) stitch them into a globally
consistent map. (d) Later, to locate the robot, we extract features from a query image, and (e) they vote for potential image poses in the map. A peak in the
voting map determines inlier features, from which (f) we recover the pose of the query image.
Location-aware applications play an increasingly critical role in everyday
life. However, the most common global localization technology Ð GPS Ð
has limited accuracy and can be unusable in dense urban areas and indoors.
We introduce an image-based global localization system that is accurate to a
few millimeters and performs reliable localization both indoors and outside.
The key idea is to capture and index distinctive local features in ground
textures. This is based on the observation that ground textures including
wood, carpet, tile, concrete, and asphalt may look random and homogeneous, but all contain cracks, scratches, or unique arrangements of carpet
fibers. These imperfections are persistent, and can serve as local features.
Our system incorporates a downward-facing camera to capture the fine texture of the ground, together with an image processing pipeline that locates
the captured texture patch in a compact database constructed offline. We
demonstrate the capability of our system to robustly, accurately, and quickly
locate test images on various types of outdoor and indoor ground surfaces.
CCS Concepts: · Computing methodologies → Image manipulation;
Computational photography;
Additional Key Words and Phrases: texture, feature, localization
ACM Reference format:
Linguang Zhang, Adam Finkelstein, and Szymon Rusinkiewicz. 2017. HighPrecision Localization Using Ground Texture. ACM Trans. Graph. 1, 1,
Article 1 (October 2017), 17 pages.
https://doi.org/10.1145/nnnnnnn.nnnnnnn
2017. 0730-0301/2017/10-ART1 $15.00
https://doi.org/10.1145/nnnnnnn.nnnnnnn
2017-10-29 16:04 page 1 (pp. 1-17)

1

INTRODUCTION

The Global Positioning System (GPS) receiver has become an essential component of both hand-held mobile devices and vehicles
of all types. Applications of GPS, however, are constrained by a
number of known limitations. A GPS receiver must have access
to unobstructed lines of sight to a minimum of four satellites, and
obscured satellites significantly jeopardize localization quality. Indoors, a GPS receiver either is slow to obtain a fix, or more likely
does not work at all. Even outdoors, under optimal circumstances,
accuracy is limited to a few meters (or perhaps a meter with modern SBAS systems). These limitations make GPS insufficient for
fine-positioning applications such as guiding a car to a precise location in a parking lot, or guiding a robot within an indoor room or
warehouse. To overcome the robustness and accuracy limitations
of GPS, alternative localization technologies have been proposed,
which are either less accurate than GPS (e.g., triangulation of cellphone towers and WiFi hotspots), or too expensive and/or cumbersome to deploy (e.g., RFID localization or special-purpose sensors
embedded in the environment). Inertial navigation and odometry,
which are often used in robotics for fine-positioning tasks, require
a known initial position, drift over time, and lose track (requiring
manual re-initialization) when the device is powered off.
This paper proposes a system that provides millimeter-scale localization, both indoors and outside on land. The key observation
ACM Transactions on Graphics, Vol. 1, No. 1, Article 1. Publication date: October 2017.

1:2 •

Zhang, L. et al.

behind our approach is that seemingly-random ground textures exhibit distinctive features that, in combination, provide a means for
unique identification. Even apparently homogeneous surfaces contain small imperfections ś cracks, scratches, or even a particular
arrangement of carpet fibers ś that are persistently identifiable as
local features. While a single feature is not likely to be unique over
a large area, the spatial relationship among a group of such features
in a small region is highly likely to be distinctive, at least up to the
uncertainty achievable with coarse localization methods such as
GPS or WiFi triangulation. Inspired by this observation, we construct a system called Micro-GPS that includes a downward-facing
camera to capture fine-scale ground textures, and an image processing unit capable of locating that texture patch in a pre-constructed
database within a few hundred milliseconds.
The use of image features for precise localization has a rich history, including works in graphics such as Photo Tourism [Snavely
et al. 2006] and Computational Re-Photography [Bae et al. 2010].
Thus, a main contribution of our work is observing that some of
the algorithms used for feature detection and matching in łnaturalž
images, as used by previous work, can also be used for łtexturelikež images of the ground. In exploring to find a particular combination of known methods that can robustly work in this way, we
also exploit two key advantages of ground-texture images. First,
the ground can be photographed from much closer range than typical features in the environment, leading to order-of-magnitude improvement in precision. Second, the statistics of texture-like images
lead to a greater density of features, leading to greater robustness
over time.
Our system consists of two phases: an offline database construction phase, and an online localization phase (Figure 1). We begin by
collecting ground texture images and aligning them using global
pose optimization. We extract local features and store them in a
database, which is subsequently compressed to a manageable size.
Next, during the localization phase, we find features in a query
image and search the database for candidate matches using approximate nearest neighbor matching. We find that in typical textures
more than 90% of the matches are spurious. We therefore use a voting method to reject outliers, based on the observation that inlier
matches will vote for a consistent location whereas outliers distribute their votes randomly. Finally, we use the remaining inlier
matches to precisely calculate the location of the query image.
The major contributions of this paper are:
• Describing a low-cost global localization system based on
ground textures and making relevant code and instructions
available for reproduction.
• Capturing and making available datasets of seven indoor and
outdoor ground textures.
• Describing an architecture for building a global map of commonplace texture, then expanding it at a later date to incorporate new data.
• Investigating the design decisions necessary for practical
matching in texture-like images, as opposed to natural images. This includes the choice of descriptor, strategies for
reducing storage costs, and a robust voting procedure that
can find inliers with high reliability.
ACM Transactions on Graphics, Vol. 1, No. 1, Article 1. Publication date: October 2017.

• Demonstrating real-world applications of precise localization, including capturing the locations of footsteps as well as
an autonomous vehicle that can use Micro-GPS to record a
path and then follow it with sub-centimeter accuracy.
Knowing the precise location of a footstep both indoors and outside offers a range of HCI and IoT applications, in addition to the
more obvious use in performance capture. Moreover, the ability
to localize a vehicle or robot precisely has the potential for farreaching applications. A car could accurately park itself (or guide
the driver to do so) in any location it recognizes from before, avoiding obstacles mere centimeters away. A continuously-updated map
of potholes could be used to guide drivers to turn slightly to the left
or right to avoid them. The technology applies equally well to vehicles smaller than cars, such as Segway, electric wheelchairs, and
mobility scooters for the elderly or disabled, any of which could be
guided to precise locations or around hard-to-see obstacles. Indoor
applications include guidance of warehouse robots and precise control over assistive robotics in the home.

2

RELATED WORK

Feature Detectors and Descriptors: There is considerable research
on finding interest points, estimating canonical scales and orientations, and computing discriminative fixed-dimensional descriptors;
these are all well-investigated problems and have inspired many
feature-based computer vision applications.
A seminal strategy for detection of interest points is the use of
image corners, found for example by the corner detector of Harris
and Stephens [1988]. Instead of detecting corners, one can alternatively detect łblobsž in the images. The SIFT detector [Lowe 2004],
which detects extrema over space and scale, has shown great success in terms of robustness and repeatability. The original SIFT
approximates the Laplacian of Gaussian (LoG) filter with a Difference of Gaussians (DoG), while SURF [Bay et al. 2006] accelerates
the process further by approximating LoG with a box filter. SIFT
assigns an orientation to each interest point based on the dominant
gradient orientation in its local image window. The scale and rotation invariance properties of SIFT make it popular in the domain
of structure from motion [Snavely et al. 2008; Wu 2011], where a
particular feature is typically observed with different scales and
orientations in different images.
Discriminative feature descriptors are crucial for uniquely identifying an image patch. SIFT [Lowe 2004] computes histograms
of gradient orientations and is robust against modest pose and illumination changes. Several subsequent descriptors offer faster
computation, including SURF [Bay et al. 2006], DAISY [Tola et al.
2010] and binary descriptors [Rublee et al. 2011; Balntas et al. 2015].
In a more recent approach, Deep Descriptors are constructed via a
Siamese convolutional neural network [Simo-Serra et al. 2015], and
outperform SIFT on some natural image datasets.
Although feature descriptors are designed to be discriminative,
the problem becomes more challenging for feature matching in
large databases; a descriptor computed from a small image patch
surrounding an interest point might not be discriminative enough.
Von Hundelshausen and Sukthankar [2012] therefore construct a
network using SIFT keypoints, and the descriptors are computed
2017-10-29 16:04 page 2 (pp. 1-17)

High-Precision Localization Using Ground Texture •
as sampled intensity values along network edges. Compared with
local patch-based descriptors, such descriptors are expected to be
more globally unique. In contrast, our approach addresses this problem by gathering votes from multiple features in an area, which collectively discriminate the location. This approach is similar to the
work on Implicit Shape Models [2004], which votes for the object
center by leveraging a pre-trained database of visual words along
with displacement vectors.
Our system relies on SIFT for feature detection and description.
While they have been employed extensively in research and practice, prior systems have generally applied SIFT to natural images.
Since natural images have different statistics from the texture-like
images studied in this paper, it is not obvious that that SIFT should
perform well for our application. However, Liu [Liu 2013] showed
that SIFT descriptors are sufficiently unique and discriminative to
be used for matching in textures like the ones we investigate. We
build upon this work by constructing a scalable system that incorporates dimensionality reduction and, most crucially, multi-feature
voting, to robustly localize within such textures. Section 6.1 offers
a comparison of SIFT, SURF, and Deep Descriptors, and shows that
SIFT outperforms the others for our datasets.
Textures for Tracking and Localization: Textures such carpet,
wood grain, concrete or asphalt all have bumps, grooves, and variations in color from location to location, and we typically use the
overall pattern or statistics of this variation to recognize a particular material. Indeed computer-based modeling and recognition of
textures traditionally proceeded along statistical lines [Dana et al.
1999; Leung and Malik 2001]. Moreover, researchers have successfully synthesized new texture by example using parametric [Heeger
and Bergen 1995] and non-parametric [Efros and Leung 1999] models. However, when we study the particular arrangement of bumps
and variations present at any location in real-world textures, we
find that it is unlikely to be repeated elsewhere.
Kelly et al. [2007] introduce a warehouse automation system in
which a downward facing camera installed on each robot is used to
help track the robot. They observe that ground surfaces usually exhibit cracks and scratches, and it is possible to track the motion of
the camera over a pre-constructed visual map. This work, however,
still assumes a known initial location and surface textures are leveraged only for pairwise (local) frame matching, much as is done in
an optical mouse. Fang et al. [2009] adopts a similar idea and uses
ICP to align the test frame with the best map frame, but this system
still relies on odometry to determine a target map frame and refine
positioning. In contrast, our approach performs global localization,
which could be used to initialize tracking systems such as these.
Clarkson et al. [2009] demonstrate that seemingly-random textures can provide a means for unique identification. The authors
observe that the fine-scale variations in the fibers of a piece of paper can be used to compute a łfingerprint" that uniquely identifies
the piece of paper. Our work demonstrates that ground textures,
including man-made ones such as carpet, share similar properties
at sufficiently fine scales, and thus may be used for localization.
Relocalization: Structure from motion allows reconstruction of a
large scale 3D point cloud offline, but relocating a newly captured
image in the reconstructed point cloud without any initial guess
2017-10-29 16:04 page 3 (pp. 1-17)

1:3

about the camera position is challenging. Previous works explore
direct 2D-to-3D matching [Sattler et al. 2011] to estimate the 6 DoF
pose of a photo with respect to a reconstructed point cloud. Li et
al. [2012] proposes a method to leverage a co-occurrence prior for
RANSAC and achieves relocalization on a larger georegistered 3D
point cloud within a few seconds. Kendall et al. [2015] trains a
convolutional neural network (PoseNet) to regress the input RGB
image to the 6-DoF camera pose. Relocalization is an essential module of modern SLAM systems, such as ORB-SLAM [Mur-Artal et al.
2015], which uses a bag-of-words model for matching.
All the above approaches except PoseNet involve large-scale feature matching, which triggers a large amount of false matches and
becomes the bottleneck of improving efficiency. To speed up feature matching, more compact models can be constructed by selecting a subset of stable 3D points from the original models [Li et al.
2010; Cao and Snavely 2014]. An effective approach to handle a
high outlier ratio is voting [Zeisl et al. 2015]. This has also proven
successful in the field of image retrieval, where spatial verification
is commonly applied to rerank the retrieved list of images, and variants of Hough voting have been proposed to improve efficiency and
robustness [Avrithis and Tolias 2014; Wu and Kashino 2015; Schönberger et al. 2016]. With more sensors available, one can utilize
the gravity direction [Svarm et al. 2016] as an additional constraint.
Baatz et al. [2010] leverages both gravity direction and a 3D scene
model to rectify images, transforming the 6-DOF pose estimation
problem into a 3-DOF problem.
Mobile devices are ideal deployment platforms for a relocalization system. Lim et al. [2015] achieve localization on a micro aerial
vehicle at interactive framerates by distributing feature extraction
over multiple frames. Middelberg et al. [2014] achieve real-time
performance by combining online camera tracking and an external
localization server. Irschara et al. [2009] and Wendel et al. [2011]
demonstrate that GPUs, which are now widely available on mobile
processors, can be used to accelerate localization.
Memory consumption quickly becomes an issue in relocalization when the offline database grows. Sattler et al. [2015] address
this problem by quantizing descriptors into a visual vocabulary so
that only visual word IDs need to be stored. Approaches based
on visual vocabulary, however, have a nontrivial łsunk costž to
store the visual words [Mikulík et al. 2010]. An alternative solution to descriptor compression is to project descriptors into a lowdimensional space. Philbin et al. [2010], for example, project the
descriptor into a 32-D space while maintaining good performance
on image retrieval. PCA-SIFT [Ke and Sukthankar 2004], despite
its name, applies Principal Components Analysis (PCA) to a normalized patch to obtain a 36-dimensional descriptor. Nevertheless,
Lynen et al. [2015] show that aggressively compressed descriptors
are weaker, although still sufficient for camera tracking.
Almost all of the above relocalization techniques rely on landmarks, such as buildings, that are normally positioned at least a few
meters from the camera. This distance, combined with finite image resolution and inherent uncertainty in camera intrinsics, means
that even a small error in feature localization results in a large uncertainty in estimated camera pose. This inaccuracy can be ameliorated by increasing the field of view of the camera [Schönbein
and Geiger 2014; Arth et al. 2011], because as more features are
ACM Transactions on Graphics, Vol. 1, No. 1, Article 1. Publication date: October 2017.

1:4 •

[I|0]

[Ri |ti ]

Zhang, L. et al.

…

[R1 |t1 ]

[Ri+1 |ti+1 ]

a

b

c

Fig. 2. Image stitching pipeline. (a) Initial stitched map, in which severe drift can be observed. (b) Loop-closure pairs found by matching neighboring frames
between columns of images. (c) Stitched map after global optimization.

detected, more constraints can be introduced to improve pose estimation. Further uncertainty comes from the ambiguity in landmark
identification, since it is not unusual to find buildings or parts of
buildings (such as windows) that appear the same from a significant distance. Moreover, natural images used by the above systems
suffer from perspective foreshortening, which brings difficulties to
feature matching. Many features are not łtime-invariantž in highly
dynamic scenes. Thus, it is necessary to update the database frequently. Finally, these systems can be affected by changes in lighting. In contrast with these systems, our work positions the camera
close to the texture being imaged and uses controlled lighting, leading to higher precision and robustness.

3

BUILDING A MAP

This section describes the offline phase of our pipeline, including
data collection, image stitching and database construction. Section 4 will describe the online phase ś the process of locating a new
frame in the pre-built map without knowing the prior frames. We
begin by describing our hardware setup, then describe the process
of building a database of features and locations.

3.1 Hardware Setup and Data Collection
Our imaging system consists of a PointGrey CM3 grayscale camera
pointed downwards at the ground (Figure 1a). A shield blocks ambient light around the portion of the ground imaged by the camera,
and a set of LED lights arranged symmetrically around the lens provides rotation-invariant illumination. The distance from the camera to the ground is set to 260 mm for most types of textures we
have experimented with. Our system is insensitive to this distance,
as long as a sufficient number of features can be detected.
The camera output is processed by an NVIDIA Jetson TX1 development board, which was chosen for both its GPU computation
capabilities (256 CUDA cores) as well as its general-purpose specifications (quad-core ARM Cortex-A57 processor, 4 GB RAM). This
processor has been used in tablets such as Google Pixel C, and it
achieves computation speed within an order of magnitude of typical desktop machines despite being mobile and battery-powered.
Our prototype has the imager and development board mounted
on a mobile cart, which may be moved manually or can be driven
with a pair of computer-controlled motorized wheels. The latter capability is used for the łautomatic path followingž demonstration
ACM Transactions on Graphics, Vol. 1, No. 1, Article 1. Publication date: October 2017.

described in Section 7. For initial data capture, however, we manually move the cart to ensure that an area can be fully covered. This
process could be automated by putting more engineering effort or
even through crowd-sourcing when there are more users. When
capturing the data, we follow a zig-zag path so that many loop
closures can be detected for global optimization. Unlike many 3D
reconstruction problems, in which the user has to carefully move
the camera and receive live feedback to guarantee full coverage,
our capture process is simple and easily mastered by non-experts.
One limitation of our prototype system is the potential for blur
caused by vibration of the cart over rough terrain. While we use
relatively short exposures (3-5 ms) to address this challenge (as well
as motion blur caused by high-speed motion) we expect that future
systems may ameliorate the vibration problem with hardware suspension and stabilization systems. Although we use a short shutter
speed, the frame rate is set to 5 fps to save disk space.
We have also tested our system on images captured on an Apple
iPhone 6 to verify that the system also works for inexpensive smartphone cameras. We use the slow-motion mode with 240 fps, which
is equivalent to a high shutter speed. In our experiments, we found
vibration to be a more critical problem for the iPhone camera, so
we put foam earplugs under the iPhone to reduce vibration.

3.2 Image Stitching
After data collection the next step is to construct a global map. We
assume that the surface is planar, even though this is not strictly
true for most outdoor surfaces. Fortunately, most ground surfaces
are locally planar, making our planar surface assumption still valid
for image stitching. Our image stitching pipeline consists of frameto-frame matching followed by global optimization, following the
general approach of Brown and Lowe [2007]. Since the computation becomes significantly more expensive as the area grows, we
use a fast method to scale image stitching to larger areas without
compromising the quality too much.
Frame-to-Frame Matching: Since the image sequence is captured
continuously by following a zig-zag path, we are able to obtain
an initial alignment of images captured. For each pair of adjacent frames, we follow a standard approach of extracting SIFT features from both frames, computing their SIFT descriptors, matching the descriptors using approximate nearest neighbors, and using
RANSAC to obtain a set of consistent matches and estimate a 2D
2017-10-29 16:04 page 4 (pp. 1-17)

High-Precision Localization Using Ground Texture •
rigid transformation. In this way, we obtain an initial global pose
for each frame, but these can be inaccurate due to accumulation of
frame-to-frame error (Figure 2a). We therefore need to use global
optimization to correct the drift.
Global Optimization: During capture, we ensure that every two
consecutive columns of images are overlapping, which provides for
many potential loop closure pairs for global optimization. In particular, for each frame we find the ten closest frames that were not
adjacent in the image sequence, given the initial global poses. We
align each candidate łloop closurež pair using the method above.
If the number of inlier feature pairs was below a threshold we discard the match; otherwise, we keep the matching features to use
during global optimization. Figure 2b shows an example of all the
loop closure pairs detected.
We perform a single global optimization, taking into account all
the detected loop closure pairs as well as the frame-to-frame pairs.
The optimization minimizes the total re-projection error of all the
matched correspondences from all the matched pairs:
Õ Õ
2
(1)
E = min
x ik − [R|t]i−1 [R|t]j · x jk
{[R |t ]}

(i, j) (x k ,x k )
i

j

where (i, j) are the indices of a pair of well-matched frames, (x ik , x jk )
is the k-th pair of matched correspondences in (i, j), and {[R|t]} is
the set of global poses of all frames. We minimize the above energy
function using the Ceres solver [Agarwal et al. 2013].
The final stitched image after global optimization is shown in
Figure 2c. Most of the errors in the initial stitched image are fixed
due to the added loop closure pairs and we almost never see blurred
regions, implying that the stitching quality is acceptable.

3.3 Scaling to Larger Areas
We have found that the above pipeline is sufficient for mapping areas up to several square meters. Beyond this, however, it becomes
challenging both to capture the data in one continuous session and
to perform the global optimization in a reasonable time. Therefore, we propose a method to capture small portions of the map at
a time, compute correspondences between these regions, and link
them together into a single global coordinate system.
Our method first asks the user to manually specify all regions
that neighbor each other. For each pair of neighboring regions, we
obtain a rough initial rigid transformation by asking the user to
click on two correspondences between the maps. Although these
correspondences could also be found by using the Micro-GPS localization system itself, as demonstrated in Section 9, in our experience
asking the user takes only a few seconds. These initial correspondences can be rough, since they will be automatically refined later.
Initial Linking: We treat each pre-stitched region as a large image and our first step is to compute the initial global poses of the
regions. We fix one region as the reference and compute the shortest path in the region-adjacency graph from the reference to all the
other regions. This is to minimize the number of pairwise transformations being concatenated so that drift can be reduced. Figure 3a
shows an example of the full map after initial linking.
2017-10-29 16:04 page 5 (pp. 1-17)

1:5

Global Optimization: We adopt the same global optimization
method we use in image stitching. For each pair of neighboring
regions, we match every pair of images across the two regions such
that the distance between the image origins is within a threshold.
We then compute a global pose for each region, taking the computed feature matches into account. Note that at this stage, the
poses of images within each already-stitched region are constant,
and we solve for only a single rigid-body transform per region.
As shown in Figure 3b, although the initial linking result is far
from perfect, our global optimization is able to correctly align the
pre-stitched regions relative to each other. However, we notice
some remaining misalignments in the overlaps between regions,
which we fix with a local refinement step.
Local Refinement: We re-optimize for the optimal pose for each
frame that overlaps two regions. These poses are optimized based
on the region-to-region correspondences computed in the above
global optimization step, as well as intra-region correspondences.
Specifically, let Ri and R j be overlapping regions, and let f be a
frame in Ri that overlaps some frames in R j . The pose [R|t]f of f is
optimized based on correspondences between f and those frames
in R j that it overlaps, as well as overlapping frames in Ri . Note
that the poses of frames that do not overlap the other region are
held fixed. Figure 3c shows a comparison of zoom-in views after
global optimization and local refinement.
Output: After global alignment, we subsample the set of images
to reduce both storage and computational costs. We remove as
many images as possible, while still ensuring that the image-space
overlap between the images we keep is at least 30%. The final output
of the image stitching pipeline is composed of the selected images
and their optimized global poses. For the dataset shown in Figure 3,
the size of the area is around 12.75m2 , and 1296 images are selected
to achieve approximately full coverage.

3.4 Database Construction
The final stage in building a map is extracting a set of features from
the images and constructing a data structure for efficiently locating
them. There are many design decisions ś which feature descriptors
to use, how many features to keep, and how much dimensionality
reduction to perform ś all having a strong impact on computational
efficiency, the size of the database, and the success rate of matching. Here we describe our actual implementation, while some of
the key decisions will be evaluated in Section 6.
Feature Database: We use the SIFT scale-space DoG detector and
gradient orientation histogram descriptor [Lowe 2004], since we
have found it to have high robustness and (with its GPU implementation) reasonable computational time. For each image in the map,
we typically find 1000 to 2000 SIFT keypoints, and randomly select
50 of them to be stored in the database. This limits the size of the
database itself, as well as the data structures used for accelerating
nearest-neighbor queries.
To further speed up computation and reduce the size of the database, we apply PCA to the set of SIFT descriptors and project each
descriptor onto the top k principal components. As described in
Section 6, for good accuracy we typically use k = 8 or k = 16 in
ACM Transactions on Graphics, Vol. 1, No. 1, Article 1. Publication date: October 2017.

1:6 •

Zhang, L. et al.

a

c

b

Fig. 3. Region linking. The full map is constructed from four pre-stitched regions. (a) Initial linking result. (b) After global optimization. (c) Close-ups of the
region marked in red, before and after local refinement.

our implementation, and there is minimal cost to using a łuniversalž PCA basis constructed from a variety of textures, rather than
a per-texture basis.
Each SIFT keypoint is associated with the image it came from
(for which we know the global pose), as well as the keypoint coordinates (x, y), orientation θ , and scale s. The feature’s pose in image
coordinates can be found from (x, y, θ ), and then transformed into
global coordinates by composing with the image pose. A feature located in the overlap between two images can potentially have two
copies, one extracted from each image. We detect this situation and
keep only the feature that was closer to the center of its image.
Building the Search Index: One of the major advantages of our
system is that the height of the camera is fixed, so that the scale of
a particular feature is also fixed. This means that when searching
for a feature with scale s in the database, we only need to check features with scale s as well. In practice, to allow some inconsistency,
we quantize scale into 10 buckets and divide the database into 10
groups based on scale. Then we build a search index for each group
using FLANN [Muja and Lowe 2009]. During testing, given a feature with scale s, we only need to search for the nearest neighbor
in the group to which s belongs.

4

LOCALIZATION

The input to our online localization phase is a single image. We
assume that the height of the camera above the ground is the same
as during mapping (or that the ratio of heights is known), so that
the image scale is consistent with the images in the database.
Feature Computation and Matching: We first extract SIFT features
from the test image as in database construction. Then we use the
same principal components computed during database construction to reduce the descriptors to dimension k. For each descriptor,
we search for the nearest neighbor using the search index we have
built (for the appropriate scale, as mentioned previously).
Precise Voting: Recall that we only keep 50 features per database
image. Therefore, we expect only a small subset of the features
found in the test image to have a correct match in the database.
ACM Transactions on Graphics, Vol. 1, No. 1, Article 1. Publication date: October 2017.

Test Image

Map

D
A
A

Map of feature locations

B

C

C

Feature
True Origin

B

D

Inlier
Outlier

Illustration of voting
Precise voting map
Fig. 4. Top left: A simple strategy would be to vote for the locations of
matched features. Right: We instead employ a łprecisež voting strategy in
which each feature votes for the location where the origin of the test image
would be located in the map, if that feature were a correct match. True
matches lead to a concentration of votes. Bottom left: The precise voting
leads to a more defined local maximum.

Finding this small set of inliers is challenging, given that standard
methods such as RANSAC do not work well if outliers highly outnumber inliers. Instead, we adopt a voting approach.
Our method is based on the observation that, due to the randomness of ground textures, false matches are usually evenly distributed
in the map. Fortunately, since true matches usually come from one
or two images, they are concentrated in a small cluster. Figure 4,
top left, shows a heat map of feature matches in a database, with
red indicating high density, green intermediate, and blue indicating low density. While this might appear promising, and indeed
we are able to build a system with modest success rates based on
this principle, the relative concentration of correct matches is still
insufficiently high relative to the density of outliers.
The problem with voting based on feature location is that the
correct features are distributed throughout the entire area corresponding to the test image. This leads to only a moderately-high
density of votes in the map near the location of the test image. The
solution is to concentrate the votes: we want all of the true features
2017-10-29 16:04 page 6 (pp. 1-17)

High-Precision Localization Using Ground Texture •
to vote for the same point in the map, leading to a much greater
difference between the peak corresponding to the true location and
the background density of outliers.
In particular, each feature casts a vote for the origin of the test
image by assuming that nearest neighbors are true matches (see
Figure 4, right). Denote a feature extracted from the test image as
ft and its nearest neighbor in the database as fd . If the feature pair
{ ft , fd } is a true match, we can compute the pose of the test image
T in world coordinates, denoted [R|t]W
T , by composing the pose of
fd in world coordinates and the pose of ft in the test image:
f

f

t
W
d
[R|t]W
T = [R|t]fd [R|t]f t [R|t]T ,

(2)

f

where [R|t]fd is assumed to be the identity. We then vote for the
t
location of the origin of the test image, which is the translational
component of [R|t]W
T .
Using this strategy, implemented via voting on a relatively fine
spatial grid, we find a much tighter peak of votes, relative to the
uniform background of outliers, as shown in Figure 4, bottom left.
Final Refinement: After voting, the cell with the highest score is
very likely to contain the true origin of the test image. We select all
of the features in that peak as likely inliers, and perform RANSAC
just on them to obtain a final estimate of the pose of the image.

5

DATASETS

We have experimented with a variety of both indoor and outdoor
datasets, covering ground types ranging from ordered (carpet) to
highly stochastic (granite), and including both the presence (concrete) and absence (asphalt) of visible large-scale variation (see Appendix for details). We will make these datasets, together with
databases of SIFT features and test-image sequences, available to
the research community. While we believe that our system is applicable to a wide range of floor and ground materials, localization
is of course only possible in the presence of unique and stable textures. We believe that our system would not work in areas containing, for example, snow, high-pile carpet, or vinyl flooring with
highly-repeating printed texture.

5.1 Indoor Datasets
Tiles: This is a dataset of man-made indoor tiles containing colored chips in a white binder. While the tiles as a whole have a
regular shape, the patterns on the tiles are random. We captured
this dataset (shown in Figure A.1) in color, using the camera of the
iPhone 6. The color of the overall map is not consistent, because
the map is stitched from four individual regions captured on different days and because we intentionally captured the lower part of
the map with no extra illumination, to test the robustness of our
system. Note that the grid lines in the stitched map are straight,
demonstrating good alignment. The accompanying video shows
the robustness of localization in this dataset, despite the variation
in color and contrast.
Carpet: This dataset contains a commercial low-pile carpet, typical of that found in many office buildings. We captured the dataset
(shown in Figure A.2) using the PointGrey camera with ambient
2017-10-29 16:04 page 7 (pp. 1-17)

1:7

light blocked and illumination from the ring of LEDs. A challenging aspect of this material is that it has a pattern that is close to
repeating, potentially making feature matching difficult. Nevertheless, with high resolution there is sufficient uniqueness everywhere
to enable feature-based localization. Moreover, as shown in Section 6.4, it remains robust to our attempts to perturb the carpet.
Wood: This is a dataset of a stained hardwood floor, as commonly
used in both residential and commercial applications (Figure A.3).
Although the wood contains significant texture, this texture consists mostly of elongated elements, unlike the łblob-likež elements
found in our other datasets. Since this is not a situation for which
the SIFT detector was designed, we detect an order of magnitude
fewer features in these images. As discussed in Section 9, this decreases the performance of our system. In an attempt to regain
some of this lost performance, we index 100, instead of 50, features
for each database image.

5.2 Outdoor Datasets
Granite: This is an outdoor dataset of large slabs of granite laid
to form a sidewalk (Figure A.4). The surface of the granite is rough,
leading to variability in shading from hour to hour and day to day,
unless ambient light is blocked.
Asphalt with Fine Aggregate: Road surfaces and sidewalks are
often coated with asphalt containing aggregate ś stones with a
particular size distribution. This dataset is of asphalt with fine aggregate on a roadway, as shown in Figure A.5. This dataset was
captured near a loading zone where trucks are parked every day,
which explains the presence of long cracks. In our stitched map,
these cracks are faithfully recovered.
In order to test the robustness of our system under different lighting, we captured two versions of this dataset: under directional
sunlight, with the cart’s light shield removed (shown in Figure A.5)
and at night, using LED illumination (Figure A.6). The results of
this experiment are described in Section 6.4.
Asphalt with Coarse Aggregate: This dataset, shown in Figure
A.7, differs in many ways from the previous one. First, it is relatively new, meaning that there is little large-scale variation from
place to place. Therefore, localization can only leverage fine-scale
texture. Second, the coarse surface caused significant vibration in
our cart, leading to occasional motion blur during capture. Nevertheless, due to our use of a short shutter speed, our reconstructed
map is sharp almost everywhere.
Concrete: Figure A.8 shows our concrete dataset. Stains on the
ground can be clearly seen in the reconstructed map. The straight
lines dividing the concrete also gives a sense of the quality of the
reconstructed map.

6

EVALUATION

In order to evaluate the accuracy and robustness of a localization
system, a typical approach would be to obtain ground-truth location and pose using a precise external measurement setup. However, this is impractical in our case due to the large areas mapped
and the precision with which we are attempting to localize. Moreover, we are more interested in repeatability, rather than absolute
ACM Transactions on Graphics, Vol. 1, No. 1, Article 1. Publication date: October 2017.

Zhang, L. et al.

Elapsed

# frames

Rate-8

Rate-16

tiles
carpet
wood
granite tiles
fine asphalt
coarse asphalt
concrete

18 days
30 days
0 days
6 days
16 days
17 days
26 days

1621
1179
311
862
651
771
797

93.83%
99.49%
59.48%
79.47%
76.04%
97.54%
83.31%

98.52%
100.00%
75.24%
93.85%
94.31%
99.74%
93.48%

90
80

50
40
30
20

The performance of our system, implementing the pipeline described in Section 4, is shown in Table 1. The two columns at right
show performance with 8-dimensional and 16-dimensional descriptors, respectively. The correct pose is recovered over 90% of the
time for most datasets (with independent per-frame localization
and no use of temporal coherence), with the exception of the wood
floor. This is because relatively few SIFT features are available in
this dataset ś see discussion in Section 9.
The following sections evaluate the impact of various design decisions, compare to image retrieval, and investigate the effect of
deliberate attempts to introduce difficult matching conditions on
the accuracy of the system.

6.1 Impact of Design Decisions
Selection of Feature Descriptor: We first evaluate the impact on
accuracy of using different feature descriptors. While SIFT [Lowe
2004] has been popular since its introduction more than a decade
ago, more recent descriptors such as SURF [Bay et al. 2006] have
been shown to achieve similar performance at lower computational
cost. Even more recently, convolutional neural networks have been
used in techniques such as Deep Descriptors [Simo-Serra et al.
2015] to achieve better matching performance. In all of these cases,
however, the performance has been optimized for natural, rather
than texture-like, images.
ACM Transactions on Graphics, Vol. 1, No. 1, Article 1. Publication date: October 2017.

SIFT
Deep (texture)
SURF
Deep (original)

70
60
50
40
30
20

10

10
4

8

16

32

64

0
2

128

4

descriptor dimensionality

8

16

32

64

128

descriptor dimensionality

Fig. 5. Comparison of different feature descriptors on both indoor and outdoor datasets. The horizontal axis indicates the dimensionality of descriptors after PCA.

success rate (%)

We therefore adopt an evaluation methodology based on comparing the query image against an image captured during mapping.
Using the pose predicted by Micro-GPS, we find the closest image
in the database, and compute feature correspondences (using all
SIFT keypoints in the image, not just the 50 used for the database).
If there are insufficiently many correspondences, we mark the localization result as a failure. We then compute a best-fit relative
pose using those features. If the pose differs by more than 30 pixels
(4.8 mm) in translation or 1.5◦ in rotation from the pose output by
Micro-GPS, we again mark the result as a failure. Finally, given a
sequence of consecutive poses that should be temporally coherent,
we detect whether the poses of any frames differ significantly from
their neighbors.

80

60

carpet

100

accuracy, given that most of the applications we envision will involve going to (or avoiding) previously-mapped locations.

90

70

0
2

fine asphalt

100

SIFT
Deep (texture)
SURF
Deep (original)

success rate (%)

Texture

carpet

100

90

80

80

70
60
50
40
30

all NNs
1600 best NNs
800 best NNs
400 best NNs
200 best NNs

20
10
0
2

4

8

16

fine asphalt

100

90

32

descriptor dimensionality

64

128

success rate (%)

Table 1. Performance of Micro-GPS. From left to right: texture type,
elapsed time between capture of database and test sequence, number of
test frames, and success rates using 8- and 16-dimensional descriptors.

success rate (%)

1:8 •

70
60
50
40
30

all NNs
1600 best NNs
800 best NNs
400 best NNs
200 best NNs

20
10
0
2

4

8

16

32

64

128

descriptor dimensionality

Fig. 6. Success rate achieved by varying the number of features that participate in voting, as well as the dimensionality of feature descriptors.

The original Deep Descriptors model provided by [Simo-Serra
et al. 2015] is trained on natural images, which does not work optimally on our texture images in our early experiments. We therefore reimplemented Deep Descriptors and retrained using patches
cropped from our datasets. Unlike the original paper, which crops
image patches around SIFT keypoints, we crop image patches from
randomly generated keypoints in the overlapping regions. This
modification prevents the trained descriptor from being biased to
a particular type of keypoint detector. We also adopted hard negative mining after pretraining, as described in the original work: we
observe that this effectively improves discriminability.
Figure 5 compares the accuracy of SIFT, SURF, and both the original and re-trained Deep Descriptors on indoor (carpet) and outdoor
(fine asphalt) datasets. In all cases, we compute the descriptors on
the same set of keypoints (extracted using the SIFT blob detector),
and evaluate performance as a function of the number of PCA dimensions retained. SURF and the original Deep Descriptor (trained
on natural images) perform poorly on both datasets, but the retrained Deep Descriptor achieves around an 85% success rate. SIFT
achieves 95-100% accuracy, while also being faster to compute than
Deep Descriptor (comparing our PyTorch implementation of Deep
Descriptor against a publicly available GPU implementation for
SIFT [Wu 2007]). Considering both performance and accuracy, we
conclude that SIFT is the most suitable descriptor for our system.
Number of Features for Voting: We next evaluate the impact of
restricting the number of features used for voting. Specifically, we
limit voting to only the n feature matches having the smallest L 2
distance between the query and database descriptors. As before,
we run our experiments while retaining different numbers of PCA
dimensions.
2017-10-29 16:04 page 8 (pp. 1-17)

High-Precision Localization Using Ground Texture •

L(di , d j ) =

(

f (di) − f (d j ) 2 ,

max 0, C − f (di ) − f (d j ) 2 ,

(i, j) is a match

(i, j) is a non-match
(3)
where (di , d j ) is a pair of descriptors, f (.) is the dimension reduction function, and C = 4 is the maximum loss of a non-match pair.
The batch size is set to 256, in which 128 are positive pairs and
the other 128 are negative pairs. After pre-training, we also adopt
hard negative mining: we only backpropagate the largest 32 losses,
which correspond to the hardest 32 pairs of descriptors.
We experimented with both random initialization and initializing
the fully connected layer with the parameters computed via PCA.
Table 2 shows that PCA outperforms the learning-based method,
even when the fully connected layer is initialized with principal
components. This is partially because the loss function only penalizes based on pairwise distance, which does not account for global
distinctiveness.
2017-10-29 16:04 page 9 (pp. 1-17)

Dimension
128
32
16
8

1-fc

1-fc (PCA)

PCA

original

34.06%
29.31%
22.24%

45.56%
43.91%
30.79%

58.72%
55.72%
31.00%

57.01%
-

carpet

100

90

80

80

70
60
50
40
30
20

4

8

16

32

64

70
60
50
40
30
20

fine asphalt
carpet
all

10
0
2

fine asphalt

100

90

success rate (%)

Method of Dimension Reduction: The effectiveness of Micro-GPS
largely relies on the global distinctiveness of the descriptor, which
can be evaluated by observing whether the true match of a descriptor can be found in a large collection of descriptors. Specifically,
given n pairs of descriptors, we first search for the two nearest
neighbors of every descriptor. We count the number of times (n true )
the second-nearest neighbor is a true match. The distinctiveness of
the descriptor can be computed as n true /2n. Unlike natural images
used by most previous relocalization systems, our data seldom suffers from perspective foreshortening, making the feature matching
process less noisy.
Dimensionality reduction must be efficient since it needs to be
performed on every online frame. Therefore we limit our consideration to linear dimensionality reduction methods, which can
be performed with just a matrix multiplication. Principal Components Analysis (PCA) is an obvious first choice, but we investigate
whether a better set of bases for dimensionality reduction can be obtained through learning. Specifically, we train one fully-connected
layer to reduce dimensionality, following the same Siamese configuration as described in the Deep Descriptor work [2015]. We first
extract pairs of matched SIFT descriptors and separate them into
training and test sets. The loss function of the network minimizes
the distance between matched descriptors and maximizes the distance between non-matched descriptors:

Table 2. Descriptor distinctiveness after dimensionality reduction using a
fully-connected perceptron layer (1-fc), using either random or PCA initialization. We find that PCA is superior to both of these approaches.

success rate (%)

Given the results in Figure 6, we observe that our system achieves
nearly perfect performance on the carpet dataset by using only 8
dimensions. The asphalt dataset requires higher descriptor dimensionality (16) to perform robustly. The number of nearest neighbors
we include in the testing phase does not seem to have a significant
influence on the performance when the descriptor dimensionality
is large enough. Including more nearest neighbors, however, improves the success rate when the descriptor dimensionality is not
sufficient (e.g. using 8 dimensions for the fine asphalt dataset). In
all cases, we found that restricting the number of features to use for
voting made little difference to computation time, since the majority of time was spent on computing the descriptors and performing
nearest-neighbor lookups.

1:9

fine asphalt
carpet
all

10
128

0
2

descriptor dimensionality

4

8

16

32

64

128

descriptor dimensionality

Fig. 7. Performing dimensionality reduction using a basis computed by applying PCA to the same dataset, a different dataset, or the concatenation
of all datasets. We see that the particular choice of PCA basis has minimal
impact on accuracy, for 8 dimensions and above.

100

Direct NN-8
VW-128-carpet

Direct NN-16
VW-128-universal

VW-128-asphalt

75

50

25

0

Carpet

Asphalt

Fig. 8. Comparing matching features using the original 8- and 16dimensional descriptors, as well as using visual words (computed from the
dataset itself, a different dataset, or the concatenation of all datasets).

Choice of PCA Basis: We next investigate whether the PCA basis
used for dimensionality reduction should be specific to each dataset,
or whether a łuniversalž basis computed from the union of a variety
of different textures achieves similar performance. For each of the
datasets shown in Figure 7, we evaluate the performance by applying the PCA basis computed from the dataset itself, the PCA basis
computed from a completely different dataset, and PCA basis computed from features extracted from the union of all the datasets. In
both figures, there are slight differences for very low-dimensional
descriptors, but the differences become negligible at 8 dimensions
and above. Given that real applications will use a dimensionality
of 8 or 16, we conclude that there is no drawback to using a single
PCA basis computed from all the datasets.
Quantization Using Visual Words: An alternative solution to
speeding up nearest-neighbor search when the database becomes
large is to quantize descriptors using visual words. Visual words
can be viewed as Voronoi cells, and descriptors belonging to the
same cell are assumed to be matched.
ACM Transactions on Graphics, Vol. 1, No. 1, Article 1. Publication date: October 2017.

1:10 •

Zhang, L. et al.
Table 3. Performance of the image retrieval system, tuned to consume the
same storage as Micro-GPS (cf. Table 1). From left to right: texture type,
dimensionality of the descriptors used by Micro-GPS (used for setting the
number of features for image retrieval, to ensure equivalent storage), and
retrieval rates achieved using 8-, 16-, and 128-bit binary signatures.

100
90

success rate (%)

80
70
60
50
40
30
coarse aspahlt
fine asphalt
carpet
fine asphalt (controlled light)
concrete

20
10
0
0.10(1.590)

0.20(0.795)

0.30(0.530)

0.40(0.397)

0.50(0.318)

0.60(0.265)

0.70(0.227)

image scale (mm/pixel)

Fig. 9. Success rate achieved by varying the scale of the image (horizontal
axis) on which the SIFT detector is run.

We generate 216 visual words by applying k-means clustering
to descriptors extracted from all the database images in a single
dataset. We also compare to visual words computed from a completely different dataset, and from the union of all the datasets. In
the results shown in Figure 8, we observe that using visual words
computed from each dataset itself (fourth column at left, third column at right) leads to better performance than using visual words
from the other dataset (third column at left, fourth column at right),
while a universal dictionary (fifth column) is somewhere in between. Nevertheless, directly using PCA dimension-reduced descriptors (first two columns) still shows better performance with
less dependence on the dataset.
Resolution: The final design decision we evaluate is what impact
image resolution has on localization performance. We expect that
for most datasets there will be a łsweet spotž of resolution: using
images that are too coarse will capture only large-scale features,
which might not be discriminative enough. Conversely, using too
fine a resolution might populate the database with tiny features
that are easily disturbed over time.
To find out the best resolution to work on, we downsample the
images with different ratios and evaluate the performance by extracting SIFT features from the downsampled images. The performance on different datasets is shown in Figure 9. We report both the
downsampling rate and actual physical resolution in mm per pixel.
In most cases, the performance is maximized when the resolution
is 0.318mm per pixel (corresponding to 0.5× downsampling relative to our original capture resolution), and some datasets exhibit a
dropoff in performance at higher resolution. Since downsampling
the image by half can significantly speed up feature extraction, reduce noise, and maximize accuracy, we apply this downsampling
as a standard component of our pipeline.

6.2 Comparison with Image Retrieval
One alternative to our feature-voting method is image retrieval:
finding the nearest image to the query in the database, and performing image-to-image alignment afterwards. We compare Micro-GPS
to a state-of-the-art image retrieval system [Tolias et al. 2013] while
taking storage, efficiency, and performance into account.
The retrieval system leverages visual words to avoid storing the
original SIFT descriptors. In our experiments, we first generate 216
visual words by clustering the descriptors using k-means. Descriptors assigned to the same visual word are then discriminated by
a binary signature computed through Hamming Embedding. The
ACM Transactions on Graphics, Vol. 1, No. 1, Article 1. Publication date: October 2017.

Texture
carpet
fine asphalt
wood

Micro-GPS
Equivalent

8-bit

8-dim
16-dim
8-dim
16-dim
8-dim
16-dim

74.39%
96.27%
26.57%
48.85%
33.44%
31.83%

Success rate
16-bit 128-bit
98.81%
99.92%
80.65%
94.16%
54.98%
54.02%

99.07%
99.92%
90.63%
97.54%
59.16%
57.56%

length of the binary signature is the same as the length of the original descriptor (i.e., 128 if SIFT is used). This results in both low
storage requirements and high efficiency in matching. Given the
SIFT descriptors computed from the test image, binary signatures
are computed and matched against the database. Each matched signature votes for the image index with which it is associated. The
image receiving the most votes is considered the łmatch.ž
Storing the binary signature as well as the location used for pose
estimation for every descriptor, costs a non-trivial amount of storage. To ensure a fair comparison against our approach, we also
apply PCA dimension reduction to the visual words and randomly
sample features from each image so that the system consumes the
same amount of storage as Micro-GPS. The dictionary of visual
words also consumes some storage, but we ignore this because the
cost is fixed and does not grow with the size of the map. The retrieval system is able to sample many more features when MicroGPS uses high-dimensional descriptors. Micro-GPS nevertheless
often only needs low-dimensional descriptors to achieve good performance, and the retrieval system does not show a strong advantage in storage in this case.
Table 3 shows the performance of the retrieval system. The retrieval system achieves the best performance when the full 128-bit
binary descriptor is used, even though this requires the use of fewer
features. In other words, given a fixed size of storage, the retrieval
system prefers better features to more, but lower-quality, features.
The performance of the retrieval system is comparable to that of
Micro-GPS on the carpet and asphalt datasets, but is considerably
worse on the wood. This is due to the significantly smaller amount
of features that can be detected on wood. In the retrieval system,
votes from correct matches are often distributed to more than one
database image, which can trigger failure when the number of features is small. In contrast, Micro-GPS utilizes all correct matches
to estimate the pose.
The system runs in single-threaded Matlab on a server, and the
average computational time on a typical dataset such as the carpet
is 348ms, excluding feature extraction and final pose estimation.
While an image retrieval system offers satisfactory performance
on common datasets, our voting-based system appears to be both
more accurate and faster for localization.
2017-10-29 16:04 page 10 (pp. 1-17)

High-Precision Localization Using Ground Texture •
Position Comparison (Outdoor - Asphalt)

Position Comparison (Indoor - Tiles)

Our Method
VisualSFM

3000

Our Method
VisualSFM

3000
2000
Y Position (mm)

2000
Y Position (mm)

1:11

1000
0
-1000

1000
0

-2000

-2000

-3000

-3000

-400 -300 -200 -100

0

100 200 300 400

X Position (mm)

Fig. 11. Introducing occlusion and perturbation. First row: introducing
occlusion by adding leaves. Second row: introducing occlusion and perturbation by scratching. The green bounding box represents success while red
represents failure

-1000

-200

-100

0

100

200

300

X Position (mm)

Fig. 10. Comparison of camera trajectories obtained using our system with
downward-facing cameras (red lines) and a state-of-the-art structure from
motion system using outward-pointing cameras (blue lines). Left: trajectories on the outdoor asphalt dataset. The distance between the trajectories is 98.8 mm on average (maximum 211.5 mm) while the mean angle
between camera poses is 0.5 degrees (maximum 1.3 degrees). Right: trajectories on the indoor tiles dataset. The mean distance is 62.9 mm (maximum 197.7 mm) and the mean anglular difference is 2.2 degrees (maximum
2.5 degrees).

6.3 Downward- vs. Outward-Facing Cameras
Many existing systems focus on localization in natural images captured by an outward-facing camera. To compare our system with
this approach to localization, we add an outward-facing camera
(identical to the one used for Micro-GPS) to our setup and we trigger both cameras simultaneously. We use the state-of-the-art VisualSFM structure from motion system [Wu 2011; Wu et al. 2011] to
recover the 3D trajectory of the outward-facing camera. In order
to compare the resulting trajectory to ours, we need to project it
onto 2D (which we do by estimating the plane that the trajectory
lies in using least squares), and to recover the relative global scale,
rotation, and translation (which we do by minimizing least-squares
distance between points taken at the same timestamps).
Figure 10 compares both trajectories for two different environments, outdoors (with asphalt ground texture) and indoors (with
the łtilesž texture). Ground truth trajectories that are accurate up
to a few millimeters are not easy to obtain, especially outdoors.
However, we observe that the trajectory of VisualSfM (in blue) is
much noisier than that of Micro-GPS, with discrepancies of many
centimeters. In contrast, the difference in estimated orientations is
small (usually below 1 degree), suggesting that both methods were
able to recover orientation successfully.
One factor that critically affects the performance of both systems
is the number of SIFT features that can be detected. Our downwardfacing camera detects an average of 1319 features per frame in the
(outdoor) asphalt sequence and 2114 features in the (indoor) tile sequence, while the outward-facing camera detects only 637 and 256
features in the same settings. More detected features typically leads
to more matched features, and hence greater localization accuracy.
Nevertheless, outward-facing cameras are commonly used for
tracking and, at the same speed of motion, are less susceptible to
motion blur than downward-facing cameras. We conclude that our
system can be used in conjunction with a tracking system based
on an outward-facing camera, with comparable additional hardware and software costs. The datasets used in this experiment can
2017-10-29 16:04 page 11 (pp. 1-17)

also serve as a benchmark for evaluating SLAM and structure-frommotion systems, and we will make them available.

6.4 Robustness
In the following section, we investigate the robustness of the proposed system by stress-testing under different real-world application scenarios.
Delay Between Mapping and Localization: Ground textures are
unlikely to completely remain unchanged, especially for those outdoors. Since the database of our system might not be updated very
frequently, it is important to test whether the system still works for
test images captured a few days or even a few weeks after the database is constructed. Table 1 shows the performance of our system
on different types of textures. It shows that our system is robust
against changes in the scene.
Occlusion and Perturbation: Two particular ways in which
ground texture can change over time are occlusion (e.g., dirt,
leaves, etc.) and perturbation of soft materials (e.g., walking on
carpet). Figure 11, top, shows frames from a sequence in which
more and more leaves are piled on a patch of concrete. Frames
outlined in green represent success, while frames outlined in red
represent failure of localization. Note that our voting procedure is
robust against a substantial amount of occlusion.
At bottom, we show frames from a sequence in which we scratch
the carpet by hand. All frames in this sequence resulted in successful localization. Note, however, that this was a heavy-duty low-pile
commercial carpet; we expect that high-pile residential carpet may
exhibit worse performance.
Motion Blur: Motion blur can easily happen when there is vibration or the camera is moving too fast. This perturbs both the
feature detection and the computed feature descriptors, making localization less accurate. To evaluate how motion blur can affect
the performance of our system, we use a robot which can run at a
roughly-constant speed and evaluate performance by varying the
speed. Unlike the previous experiments, we set the shutter speed
to 10 ms (2 to 3 times longer than usual) to deliberately introduce
more motion blur. Examples of images captured under different
speeds are shown in Figure 12, top.
Figure 12, bottom, shows the performance of our system under
different robot speeds. When lower-dimensional descriptors are
used, we find that the performance drops quickly as the speed increases. However, if we are willing to increase the feature dimensionality, the system can handle more motion blur.
ACM Transactions on Graphics, Vol. 1, No. 1, Article 1. Publication date: October 2017.

1:12 •

Zhang, L. et al.
Table 4. Computational time breakdown for localization using 8- and 16dimensional descriptors, measured on the carpet dataset.

Operation
95 mm/s

190 mm/s

380 mm/s

570 mm/s

100
90

success rate (%)

80
70
60
50

Time-8 (ms)

Time-16 (ms)

SIFT extraction
NN Search
Voting
Pose Estimation
Others

61.30
109.64
18.69
10.44
45.51

67.52
119.64
18.68
12.09
43.63

Total

245.58

261.56

40
30
20
10

dimensionality = 16
dimensionality = 8

0

100

200

300

400

500

600

robot speed (mm/s)

Fig. 12. Top: Examples of motion-blurred images captured on the carpet.
Bottom: Success rate achieved by varying the speed of the robot. The two
curves correspond to feature dimensionality of 16 and 8, respectively. Note
that this experiment uses a shutter speed of 10 ms, as opposed to the 3ś5 ms
we typically use.

Fig. 13. Coarse asphalt after rain (compare to Figure A.7). The added specularity causes our localization system to fail.

Effectiveness at Night: As opposed to differences in intensity in
the image as a whole, as considered in the previous experiment, we
have observed that changing the pattern of illumination results in
a dramatic decrease in localization performance. In particular, we
conducted an experiment in which we captured a dataset (fine asphalt) under direct illumination from sunlight (i.e., without the light
shield shown in Figure 1), then captured our testing dataset at night,
with illumination provided by LEDs. Because of the difference between directional sunlight and LED illumination, the features that
were extracted in the two datasets were completely different, since
a majority of the features in this surface are due to shading, not
color variation. As a result, we were able to find almost no matches
between the query images and the database.
To resolve this issue, we captured a second map (Figure A.6) with
the light shade in place and using LEDs to illuminate the ground
all the time, even during daylight. Comparing this dataset to Figure A.5, we see a significant difference in the appearance of smallscale features in the closeup at lower left. Localizing our nighttimecaptured test set in our daytime-captured but light-controlled map,
we achieve a localization success rate of 94.50%, showing that blocking the environment light is an effective way to guarantee the consistency of test images and database images.
Wet Surfaces: Another change that can cause dramatic differences between features extracted at different times is wetting of
the surface. Figure 13 shows the same surface as Figure A.7, but
ACM Transactions on Graphics, Vol. 1, No. 1, Article 1. Publication date: October 2017.

captured after a rainfall. The difference in surface specularity, combined with the bumpiness of the surface, caused completely different features to be detected, leading to almost complete failure of
our system. (In contrast, we obtained very good results matching
wet concrete to a dry-concrete database.)

6.5 Efficiency
Our system is implemented on a battery-powered mobile platform
having computing power an order of magnitude lower than a desktop computer. In our implementation, we use SiftGPU [Wu 2007] to
speed up the extraction of SIFT features. We evaluate the efficiency
on NVIDIA Jetson TX1, and the computational time breakdown is
shown in Table 4. Although the system does not run in real time,
it nevertheless delivers localization in under a second. This is sufficient to correct drift in odometry, and to re-localize the system
when tracking or odometry is lost.

7

APPLICATION: AUTOMATIC PATH FOLLOWING

Our system provides a simple, inexpensive solution to achieve fine
absolute positioning, and mobile robots having such a requirement
represent an ideal application. In practice, we believe that indoor
service robots, human-free warehouse robots, and self-driving cars
performing fine positioning tasks can benefit from our system.
To demonstrate the practicality of this approach, we build a robot that is able to follow a designed path exactly without any initialization of the position. Our robot (shown in Figure 14a) has a
differential drive composed of two 24 V DC geared motors with
encoders for closed-loop control of the motors. A Teensy Arduino
board is used for the motion control. Using the encoder readings,
we implemented dead-reckoning odometry on board so that it is
feasible to track the position of the robot at reasonable accuracy
within a short distance.
The drift in odometry is corrected using Micro-GPS running on
the on-board NVIDIA Jetson TX1 computer, which communicates
with the microcontroller via a USB connection. When a video frame
is captured, the microcontroller is notified to save its current estimate of position, and to continue tracking the robot using dead
reckoning. When Micro-GPS computation is finished, the groundtruth location of the frame is communicated to the microcontroller,
which can update the estimate of its current location by applying
the intervening tracking results to that ground-truth position.
To test the repeatability of navigation using this strategy, we
first manually drive the robot along a particular path (shown in
Figure 14b), and mark its final location on the ground using a piece
2017-10-29 16:04 page 12 (pp. 1-17)

High-Precision Localization Using Ground Texture •

1:13

NVIDIA Jetson TX1
(Micro-GPS)

Manual

Manual

Auto

Auto

Camera
Microcontroller

a

b

c

d

Fig. 14. A demonstration of path following. (a) Micro-GPS is implemented as a component of a mobile robot. (b) We generate a path by manually driving
the robot. (c) We then use Micro-GPS to repeat the path. Screen-shots captured under manual and automatic driving modes are highly consistent. (d) The
robot reaches the same ending position with high accuracy.

of tape. We then send the robot back to its starting position, and
ask it to re-play that same path. The sequences of the manual driving and automatic re-play are shown in the accompanying video;
screen-shots from the video are compared in Figure 14c. As shown
in Figure 14d, the robot ends up in almost exactly the same position
after automatic path following as it did after the manual driving.

8

APPLICATION: CAPTURE OF FOOT PLACEMENT

The applicability of our system is not limited to the field of robotics. Micro-GPS can enable many more applications when tiny,
lightweight, wearable cameras foretold by GoPro become prevalent.
One of the possibilities is precise recording of foot placement, for
performance capture as well as a range of HCI and IoT applications.
We attach an iPhone low on a person’s leg to capture ground
images around the foot (Figure 15a). An ArUco marker [MunozSalinas 2012] is fixed on the shoe to help in estimating the orientation of the camera. We calibrate the intrinsics of the camera and its
pose relative to another marker on the ground.
We record video at normal shutter speed while the person walks.
The camera pose and metric scale of each frame can be estimated
via the captured marker on the shoe. We warp the image to remove
perspective foreshortening and match the scale of the database images (Figure 15b). Note that this correction cannot be performed
when the person is moving, because the calibration is for a foot
placed on the ground. Finally, Micro-GPS can be directly applied
to the warped images, and Figure 15c shows that every foot step is
successfully localized (to millimeter-level accuracy) in the map.

9

a

b

c

Fig. 15. Recording footprints. (a) The marker on the shoe is calibrated relative to the marker on the ground. (b) Captured image after performing
perspective correction. (c) Recorded footprints.

Limitations in Feature Extraction: We have found (Table 1) that
using SIFT features in the proposed system results in robust performance in most cases. One exception is the wood dataset, in which
features are more difficult to find. In particular, the number of extracted features per frame in the wood dataset can be as low as 81
(Figure 16a), as compared to over 2000 on the carpet dataset. This
is because the textures on wood are mainly composed of stripes,
not łblobs.ž

DISCUSSION, LIMITATIONS, AND FUTURE WORK

We present a low cost texture-based global positioning system that
can provide millimeter-level precision. Our system does not require
adding infrastructure to the environment, and we demonstrate that
it can be implemented on a low-power platform such as those embedded in mobile vehicles. A variety of applications that require
fine global positioning can benefit from the proposed approach.
2017-10-29 16:04 page 13 (pp. 1-17)

Fig. 16. Performance on the wood floor is limited by the lack of SIFT features (left). As future work, we suggest using (a subset of) the detected
edges instead (right).
ACM Transactions on Graphics, Vol. 1, No. 1, Article 1. Publication date: October 2017.

1:14 •

Zhang, L. et al.
a new globally consistent map (shown at left) of only the newlyacquired data. We then run Micro-GPS on each test image and register the new map to the existing map once Micro-GPS confirms a
successful localization. Lastly, we find more correspondences between the existing map and the new map and optimize the test
image poses again. The updated map after registering the new map
to the original map is shown in Figure 17, right.

Fig. 17. Map expansion. Left: Self-consistent map generated by stitching
the test sequence. Right: The updated map generated by registering the
new sub-map to the existing map.

This difficulty in feature detection leads to lower performance.
Although we report a success rate of 75% in Table 1, we have determined by hand that this is an under-estimate due to not having
enough features for verification ś the true success rate is 88%. Still,
as future work we would like to explore combining the SIFT detector with others, such as those based on Canny edges (Figure 16b).
Accuracy: The actual localization in global space is determined
by the alignment of the database images. Most of the maps we have
obtained are visually plausible and of high quality because of the
global optimization process which utilizes a large number of loopclosure pairs. However, even with global optimization, the alignment can still drift slightly as the map grows. A solution to that is
to put sparse visual anchors in the map and incorporate these anchors into optimization as hard constraints. On the other hand, in
most applications it is sufficient to locate the live frame in the prebuilt map, even though the pre-built map does not perfectly match
the real world. For example, when a warehouse robot is asked to
navigate to a shelf in the map, driving the robot until it reaches the
destination in the map is more important than knowing the location of the shelf in the real world.
Map Expansion: A long-term goal of this work is to enable crowdsourced building and updating of large-scale maps. Although it
will be difficult to apply the existing global registration methods at
global scale, this is not necessary if any rough location estimate is
available. For example, outdoors it should be possible to obtain a
GPS estimate accurate to, say, 10 m under almost all circumstances.
Indoors, WiFi triangulation can be used to obtain an initial position
estimate accurate to a few tens of meters. Therefore, each łmapž
need only cover an area of a few hundred square meters, which is
well within the practical range of our system given that we perform dimensionality reduction on features and accelerate nearestneighbor queries.
The remaining challenge is therefore a systems-building one: developing a back-end that will enable newly-acquired images to be
incorporated into the database. We have conducted an initial experiment to evaluate the practicality of aligning a new set of images
and optimizing their poses relative to an existing database, as long
as there is some overlap between the new dataset and the existing
map. Figure 17 shows a demonstration on the concrete dataset. We
first use the image stitching pipeline detailed in Section 3 to obtain
ACM Transactions on Graphics, Vol. 1, No. 1, Article 1. Publication date: October 2017.

Efficiency: SIFT feature extraction makes our system far from
real-time when implemented on a mobile platform, even with the
GPU implementation. Replacing SIFT with a much more efficient
feature detector can probably help. In particular, we would like to
investigate deep-network-based approaches that are specialized to
the domain of texture-like images, as opposed to the existing work
on natural images.
Robustness: While we have shown that our system is robust to
perturbations in both image capturing and changes to the texture,
it does not work well with wet surfaces because the appearances
of wet surfaces often change significantly. Features extracted from
wet surface images can be completely different from those extracted
from dry surface images, which is an even worse case than severe
occlusion. A possible future direction is to capture the geometry
(i.e., normal maps) of the ground texture in a manner similar to
the previous work on fingerprinting blank paper [Clarkson et al.
2009]. Another possibility would be to use cross-polarized lighting
to eliminate the specular reflections.
In our current pipeline, we build our database by randomly selecting 50 features from each database image. However, some of
the features might be non-repeatable and cannot be observed again.
Note that features with higher LoG response are not necessarily
highly repeatable features: they are just as likely to be due to noise,
dust, etc. Intuitively, features with medium responses are likely to
be highly repeatable. Hartmann et al. [2014] train a classifier to
select repeatable SIFT features, but we have found that the correlation between a SIFT descriptor and its repeatability score using this
method is not strong on our łtexture-like" images. In the future,
we would like to investigate predicting repeatable features based
on the image patches, and to include only those in the database.

REFERENCES
Sameer Agarwal, Keir Mierle, et al. 2013. Ceres Solver. http://ceres-solver.org/. (2013).
Clemens Arth, Manfred Klopschitz, Gerhard Reitmayr, and Dieter Schmalstieg. 2011.
Real-Time Self-Localization from Panoramic Images on Mobile Devices. In Proc.
ISMAR. 37ś46.
Yannis Avrithis and Giorgos Tolias. 2014. Hough Pyramid Matching: Speeded-Up Geometry Re-Ranking for Large Scale Image Retrieval. IJCV 107, 1 (March 2014),
1ś19.
Georges Baatz, Kevin Köser, David Chen, Radek Grzeszczuk, and Marc Pollefeys. 2010.
Handling Urban Location Recognition as a 2D Homothetic Problem. In Proc. ECCV.
266ś279.
Soonmin Bae, Aseem Agarwala, and Frédo Durand. 2010. Computational RePhotography. ACM Trans. Graphics 29, 3, Article 24 (July 2010), 15 pages.
Vassileios Balntas, Lilian Tang, and Krystian Mikolajczyk. 2015. BOLD ś Binary Online
Learned Descriptor For Efficient Image Matching. In Proc. CVPR.
Herbert Bay, Tinne Tuytelaars, and Luc Van Gool. 2006. SURF: Speeded Up Robust
Features. In Proc. ECCV. 404ś417.
Matthew Brown and David G Lowe. 2007. Automatic Panoramic Image Stitching Using
Invariant Features. IJCV 74, 1 (Aug. 2007), 59ś73.
Song Cao and Noah Snavely. 2014. Minimal Scene Descriptions from Structure from
Motion Models. In Proc. CVPR. 461ś468.
2017-10-29 16:04 page 14 (pp. 1-17)

High-Precision Localization Using Ground Texture •
William Clarkson, Tim Weyrich, Adam Finkelstein, Nadia Heninger, J Alex Halderman, and Edward W Felten. 2009. Fingerprinting Blank Paper Using Commodity
Scanners. In Proc. IEEE Symposium on Security and Privacy. 301ś314.
Kristin Dana, Bram van Ginneken, Shree Nayar, and Jan Koenderink. 1999. Reflectance
and Texture of Real-World Surfaces. ACM Trans. Graphics 18, 1 (Jan. 1999), 1ś34.
Alexei Efros and Thomas Leung. 1999. Texture Synthesis by Non-Parametric Sampling.
In Proc. ICCV. 1033ś1038.
Hui Fang, Ming Yang, Ruqing Yang, and Chunxiang Wang. 2009. Ground-TextureBased Localization for Intelligent Vehicles. IEEE Trans. Intelligent Transportation
Systems 10, 3 (Sept. 2009), 463ś468.
Chris Harris and Mike Stephens. 1988. A Combined Corner and Edge Detector. In Proc.
Alvey Vision Conference. 147ś151.
Wilfried Hartmann, Michal Havlena, and Kaspar Schindler. 2014. Predicting Matchability. In Proc. CVPR. 9ś16.
David Heeger and James Bergen. 1995. Pyramid-Based Texture Analysis/Synthesis. In
Proc. ACM SIGGRAPH. 229ś238.
Arnold Irschara, Christopher Zach, Jan-Michael Frahm, and Horst Bischof. 2009. From
Structure-from-Motion Point Clouds to Fast Location Recognition. In Proc. CVPR.
2599ś2606.
Yan Ke and Rahul Sukthankar. 2004. PCA-SIFT: A More Distinctive Representation for
Local Image Descriptors. In Proc. CVPR. 506ś513.
Alonzo Kelly, Bryan Nagy, David Stager, and Ranjith Unnikrishnan. 2007. An
Infrastructure-Free Automated Guided Vehicle Based on Computer Vision. IEEE
Robotics & Automation Magazine 14, 3 (Sept. 2007), 24ś34.
Alex Kendall, Matthew Grimes, and Roberto Cipolla. 2015. PoseNet: A Convolutional
Network for Real-Time 6-DOF Camera Relocalization. In Proc. ICCV. 2938ś2946.
Bastian Leibe, Ales Leonardis, and Bernt Schiele. 2004. Combined Object Categorization and Segmentation with an Implicit Shape Model. In ECCV Workshop on
Statistical Learning in Computer Vision.
Thomas Leung and Jitendra Malik. 2001. Representing and Recognizing the Visual
Appearance of Materials Using Three-Dimensional Textons. IJCV 43, 1 (June 2001),
29ś44.
Yunpeng Li, Noah Snavely, and Daniel Huttenlocher. 2010. Location Recognition Using
Prioritized Feature Matching. In Proc. ECCV. 791ś804.
Yunpeng Li, Noah Snavely, Dan Huttenlocher, and Pascal Fua. 2012. Worldwide Pose
Estimation Using 3D Point Clouds. In Proc. ECCV. 15ś29.
Hyon Lim, Sudipta N Sinha, Michael F Cohen, Matt Uyttendaele, and H Jin Kim. 2015.
Real-Time Monocular Image-Based 6-DoF Localization. International Journal of
Robotics Research 34, 4-5 (April 2015), 476ś492.
Siyu Liu. 2013. Localization Using Feature Matching in Near-Random Textures. Master’s
thesis. Princeton University.
David G Lowe. 2004. Distinctive Image Features from Scale-Invariant Keypoints. IJCV
60, 2 (Nov. 2004), 91ś110.
Simon Lynen, Torsten Sattler, Michael Bosse, Joel A Hesch, Marc Pollefeys, and Roland
Siegwart. 2015. Get Out of My Lab: Large-Scale, Real-Time Visual-Inertial Localization. In Robotics: Science and Systems XI.
Sven Middelberg, Torsten Sattler, Ole Untzelmann, and Leif Kobbelt. 2014. Scalable
6-DoF Localization on Mobile Devices. In Proc. ECCV. 268ś283.
Andrej Mikulík, Michal Perdoch, Ondřej Chum, and Jiří Matas. 2010. Learning a Fine
Vocabulary. In Proc. ECCV. 1ś14.
Marius Muja and David G Lowe. 2009. Fast Approximate Nearest Neighbors with
Automatic Algorithm Configuration.. In Proc. VISAPP. 331ś340.
Rafael Munoz-Salinas. 2012. ArUco: A Minimal Library for Augmented Reality Applications Based on OpenCV. https://www.uco.es/investiga/grupos/ava/node/26.
(2012).
Raul Mur-Artal, JMM Montiel, and Juan D Tardós. 2015. ORB-SLAM: A Versatile and
Accurate Monocular SLAM System. IEEE Trans. Robotics 31, 5 (Oct. 2015), 1147ś
1163.
James Philbin, Michael Isard, Josef Sivic, and Andrew Zisserman. 2010. Descriptor
Learning for Efficient Retrieval. In Proc. ECCV. 677ś691.
Ethan Rublee, Vincent Rabaud, Kurt Konolige, and Gary Bradski. 2011. ORB: An Efficient Alternative to SIFT or SURF. In Proc. ICCV.
Torsten Sattler, Michal Havlena, Filip Radenovic, Konrad Schindler, and Marc Pollefeys.
2015. Hyperpoints and Fine Vocabularies for Large-Scale Location Recognition. In
Proc. ICCV. 2102ś2110.
Torsten Sattler, Bastian Leibe, and Leif Kobbelt. 2011. Fast Image-Based Localization
Using Direct 2D-to-3D Matching. In Proc. ICCV. 667ś674.
Miriam Schönbein and Andreas Geiger. 2014. Omnidirectional 3D Reconstruction in
Augmented Manhattan Worlds. In Proc. IROS. 716ś723.
Johannes L Schönberger, True Price, Torsten Sattler, Jan-Michael Frahm, and Marc
Pollefeys. 2016. A Vote-and-Verify Strategy for Fast Spatial Verification in Image
Retrieval. In Proc. ACCV.
Edgar Simo-Serra, Eduard Trulls, Luis Ferraz, Iasonas Kokkinos, Pascal Fua, and
Francesc Moreno-Noguer. 2015. Discriminative Learning of Deep Convolutional
Feature Point Descriptors. In Proc. ICCV.

2017-10-29 16:04 page 15 (pp. 1-17)

1:15

Noah Snavely, Steven M. Seitz, and Richard Szeliski. 2006. Photo Tourism: Exploring
Photo Collections in 3D. ACM Trans. Graphics 25, 3 (July 2006), 835ś846.
Noah Snavely, Steven M Seitz, and Richard Szeliski. 2008. Modeling the World from
Internet Photo Collections. IJCV 80, 2 (Nov. 2008), 189ś210.
Linus Svarm, Olof Enqvist, Fredrik Kahl, and Magnus Oskarsson. 2016. City-Scale
Localization for Cameras with Known Vertical Direction. IEEE Trans. PAMI (2016).
Engin Tola, Vincent Lepetit, and Pascal Fua. 2010. DAISY: An Efficient Dense Descriptor Applied to Wide-Baseline Stereo. IEEE Trans. PAMI 32, 5 (May 2010), 815ś830.
Giorgos Tolias, Yannis Avrithis, and Hervé Jégou. 2013. To Aggregate or Not To Aggregate: Selective Match Kernels for Image Search. In Proc. ICCV. 1401ś1408.
Felix Von Hundelshausen and Rahul Sukthankar. 2012. D-Nets: Beyond Patch-Based
Image Descriptors. In Proc. CVPR.
Andreas Wendel, Arnold Irschara, and Horst Bischof. 2011. Natural landmark-based
monocular localization for MAVs. In Proc. ICRA. 5792ś5799.
Changchang Wu. 2007. SiftGPU: A GPU Implementation of Scale Invariant Feature
Transform (SIFT). http://www.cs.unc.edu/~ccwu/siftgpu/. (2007).
Changchang Wu. 2011. VisualSFM: A Visual Structure from Motion System.
http://ccwu.me/vsfm/. (2011).
Changchang Wu, Sameer Agarwal, Brian Curless, and Steven M Seitz. 2011. Multicore
bundle adjustment. In Computer Vision and Pattern Recognition (CVPR), 2011 IEEE
Conference on. IEEE, 3057ś3064.
Xiaomeng Wu and Kunio Kashino. 2015. Adaptive Dither Voting for Robust Spatial
Verification. In Proc. ICCV. 1877ś1885.
Bernhard Zeisl, Torsten Sattler, and Marc Pollefeys. 2015. Camera Pose Voting for
Large-Scale Image-Based Localization. In Proc. ICCV. 2704ś2712.

ACM Transactions on Graphics, Vol. 1, No. 1, Article 1. Publication date: October 2017.

1:16 •

Zhang, L. et al.

A APPENDIX: DETAIL OF DATA SETS
The figures below show overviews and close-ups of the indoor and
outdoor datasets that we have gathered. In each figure, we show:
• Top left: An overview of the scene where the images are captured.
• Bottom left: An example of one captured image.
• Right: A portion of the reconstructed map, with the full
stitched map shown in the inset at bottom right.

Fig. A.1. Indoor tiles. The map includes 1296 images covering approximately 1.5 × 8.5 m.

Fig. A.3. Indoor wood floor. This dataset contains 3826 images covering
approximately 8.7 × 4.8 m.

Fig. A.2. Indoor carpet. This dataset contains 2014 images covering approximately 1.5 × 11.8 m.

Fig. A.4. Outdoor granite tiles. This dataset contains 1229 images covering
approximately 8.6 × 3.2 m.

ACM Transactions on Graphics, Vol. 1, No. 1, Article 1. Publication date: October 2017.

2017-10-29 16:04 page 16 (pp. 1-17)

High-Precision Localization Using Ground Texture •

1:17

Fig. A.5. Outdoor asphalt with fine aggregate, captured under ambient illumination (sunlight). This dataset contains 2215 images covering approximately 2.6 × 7.6 m.

Fig. A.7. Outdoor asphalt with coarse aggregate. This dataset contains
2061 images and the dimension is approximately 2.0 × 10.6 m.

Fig. A.6. Outdoor asphalt with fine aggregate captured at night under controlled LED illumination. This dataset contains 2118 images covering approximately the same area as above.

Fig. A.8. Outdoor concrete. This dataset contains 3316 images and the dimension is approximately 7.6 × 4.3 m.

2017-10-29 16:04 page 17 (pp. 1-17)

ACM Transactions on Graphics, Vol. 1, No. 1, Article 1. Publication date: October 2017.

