title: Getting the distance between two geopandas data frames geometry points
body: <p>I am working with spatial data for the first time.  I have to compare two dataframes that has lat and long details.  I have converted both to GeoPandas dataframes.</p>
  <pre><code>import pandas as pd
  from pandas import DataFrame
  import geopandas as gpd
  from neighbors import nearest_neighbor
  
  
  df = pd.DataFrame([[1973,22.525158,88.330775],[1976,72.85136,19.10840],[898,91.78523,26.15012]],columns=['id', 'lat', 'long'])
  gdf1 = gpd.GeoDataFrame(df, geometry=gpd.points_from_xy(df.long,df.lat))
  
  df2 = pd.DataFrame([['06c979eaa59f',29.873870,76.965620],['19aedbb2e743',20.087574,76.180045],['5060a3931a43',31.289770,75.572340]],columns=['id','lat','lon']) 
  gdf2 = gpd.GeoDataFrame(df2, geometry=gpd.points_from_xy(df2.lon,df2.lat))
  </code></pre>
  <p>My DF1 has 1 million rows and df2 has around 7000 rows.  I am trying to get the nearest neighbors from DF2 for each record in DF1.</p>
  <p>I have tried two methods.  Both runs very fast and the results workable.  However, they are not accurate.</p>
  <p><strong>Method 1:</strong></p>
  <p><a href="https://automating-gis-processes.github.io/site/notebooks/L3/nearest-neighbor-faster.html" rel="nofollow noreferrer">Please check this link</a></p>
  <p>In this page, I have used the nearest neighbors method from <code>sklearn.neighbors</code>.  This returns the results in meters.  However, when I manually check the distance between the lat long from two data frames, I always finds the nearest neighbor returns 1/4 the distance.</p>
  <p>For example if the distance returned by the above method is 125 meters, both google map and <a href="https://www.geodatasource.com/distance-calculator" rel="nofollow noreferrer">https://www.geodatasource.com/distance-calculator</a> returns a distance of around 500 meters.  The difference in distance keeps fluctuating around 4 times of the returned result.</p>
  <p><strong>Method 2:</strong></p>
  <p>In the second method I followed the code given in gis.stackexchange.com.</p>
  <pre><code>https://gis.stackexchange.com/questions/222315/geopandas-find-nearest-point-in-other-dataframe
  
  import itertools
  from operator import itemgetter
  
  import geopandas as gpd
  import numpy as np
  import pandas as pd
  
  from scipy.spatial import cKDTree
  from shapely.geometry import Point, LineString
  
  df = pd.DataFrame([[1973,22.525158,88.330775],[1976,72.85136,19.10840],[898,91.78523,26.15012]],columns=['id', 'lat', 'long'])
  gdf1 = gpd.GeoDataFrame(df, geometry=gpd.points_from_xy(df.long,df.lat))
  
  df2 = pd.DataFrame([['06c979eaa59f',29.873870,76.965620],['19aedbb2e743',20.087574,76.180045],['5060a3931a43',31.289770,75.572340]],columns=['id','lat','lon']) 
  gdf2 = gpd.GeoDataFrame(df2, geometry=gpd.points_from_xy(df2.lon,df2.lat))
  </code></pre>
  <p>In this, I replaced the gpd1 and gpd2 with my own data frames.</p>
  <pre><code>def ckdnearest(gdfA, gdfB, gdfB_cols=['id']):
      # resetting the index of gdfA and gdfB here.
      gdfA = gdfA.reset_index(drop=True)
      gdfB = gdfB.reset_index(drop=True)
      A = np.concatenate(
          [np.array(geom.coords) for geom in gdfA.geometry.to_list()])
      B = [np.array(geom.coords) for geom in gdfB.geometry.to_list()]
      B_ix = tuple(itertools.chain.from_iterable(
          [itertools.repeat(i, x) for i, x in enumerate(list(map(len, B)))]))
      B = np.concatenate(B)
      ckd_tree = cKDTree(B)
      dist, idx = ckd_tree.query(A, k=1)
      idx = itemgetter(*idx)(B_ix)
      gdf = pd.concat(
          [gdfA, gdfB.loc[idx, gdfB_cols].reset_index(drop=True),
           pd.Series(dist, name='dist')], axis=1)
      return gdf
  
  c = ckdnearest(gdf1, gdf2)
  </code></pre>
  <p>The above runs very fast and returns the result.  However the returned distance values are atleast 100 times lower than that I get.</p>
  <p>multiplier : 107.655914</p>
  <p><a href="https://i.stack.imgur.com/SnfFa.png" rel="nofollow noreferrer"><img src="https://i.stack.imgur.com/SnfFa.png" alt="enter image description here" /></a></p>
  <p>In the above excel pic, the first column indicates the results returned by python, while the second column the results returned by the same website given above.  While these approximations in results gets me started, I want accurate results.  How do I compare the two data frames given above and get the most accurate nearest distance for each row in DF1.</p>