Order a list of points by closest distance
I don't like any of these answers.  In particular, the accepted answer is simply wrong.

Let's start by critiquing the interface:

List&lt;Point&gt; OrderByDistance(List&lt;Point&gt; pointList)


The contract is: the list must contain at least one element, the list is destroyed (!!!) by the method, the first element is special, and the result is a mutable list.  I like nothing about any of this; this sounds like a ball of potential bugs.  The right contract is:

ImmutableList&lt;Point&gt; OrderByDistance(Point start, ImmutableSet&lt;Point&gt; points)


Look at how many problems this solves. Do we need the set to contain at least one point?  No. We already have the start point. Do we destroy the set of points?  No. It's immutable.  And so on.

Now that we have the signature correct, the algorithm is simple:

ImmutableList&lt;Point&gt; OrderByDistance(Point start, ImmutableSet&lt;Point&gt; points)
{
  var current = start;
  var remaining = points;
  var path = ImmutableList&lt;Point&gt;.Empty.Add(start);
  while(!remaining.IsEmpty)
  {
    var next = Closest(current, remaining);
    path = path.Add(next);
    remaining = remaining.Remove(next);
    current = next;
  }
  return path;
}


Now all you have to do is efficiently implement Closest, which you should be able to do given the previous hints about computing distances cheaply and so on.

If you want a more sophisticated algorithm for Closest, then you need to do some research on this well-studied problem:

https://en.wikipedia.org/wiki/Nearest_neighbor_search