How to split a large set of coordinates into smaller sets using something like nearby selection?
Have a look into KD-Trees. They work by partitioning your space into discrete blocks, via a (wait for it...) tree. Here is the wikipedia example for a 3d space:



Each of the cuboids is represented by a leaf or node in the tree. It works via binary splits (splitting the space into two via a criterion that you could choose yourself). Here is a short intro to the algorithm.

There is an implementation in Scikit-Learn. You build a tree and then you can query the data for a given point and radius, returning all points within that radius/distance.

This has the benefit that you could also test your algorithms over a group of subsets by querying a selection of coordinates - and you only need to construct the tree once.



Small demo

I will generate some random points random points on a 2d integer grid, so values between 0 and 10. Some points on the grid could therefore be empty.

I put them into a Pandas Dataframe just to make things a little easier to manage and plot:

In [1]: import numpy as np                                                      

In [2]: import pandas as pd                                                     

In [3]: from sklearn.neighbors import KDTree         # could use cKDTree                           

In [4]: import matplotlib.pyplot as plt                                         

In [5]: data = {"x": np.random.randint(0, 11, 200), "y": np.random.randint(0, 10, 200)}    # 200 random coords                                    

In [6]: df = pd.DataFrame(data)                                                 

In [7]: df.head()                                                               
Out[7]: 
   x  y
0  1  7
1  9  8
2  6  1
3  3  2
4  4  3


Now I create the KDTree and query the coordinate (5, 5) in the centre - I want all points back that lie within a radius distance of 3:

In [8]: tree = KDTree(df.values)

In [9]: ix = tree.query_radius([(5, 5)], r=3)[0] 


Now I use the returned indices (ix) to filter out those points I want - and just plot them here in red, along with the original complete dataset in blue:

In [10]: ax = df.plot.scatter("x", "y", c="b", alpha=0.5);   # blue base points

In [11]: df.iloc[ix].plot.scatter("x", "y", c="r", ax=ax);   # query results

In [12]: plt.scatter(5, 5, c="g");    # query point

In [11]: plt.show()



The queried point is green (5, 5)
Original points are blue
Query results are red




Faster please!

There is also the cKDTree class, which works very similarly, but is implemented in C and so should be faster in many cases. Have a look here at some differences. the cKDTree is the best choice if you want to do simple querying like in my example!