Program an Uncircularness Score
Python 2 with numpy, 116 bytes

from numpy import*
def f(x,y):a=linalg.lstsq(hstack((x,y,ones_like(x))),(x*x+y*y)/2);return a[1]/sum((x-a[0][0])**4)


Takes x and y as 2d column vectors and returns an array containing the answer.  Note that this will give an empty array for a perfectly straight line or with 3 or fewer points.  I think lstsq gives no residuals if there's a perfect fit.

Explanation

Essentially, this finds the circle of best fit and gets the squared residuals.

We want to minimize (x - x_center)^2 + (y - y_center)^2 - R^2.  It looks nasty and nonlinear, but we can rewrite that as x_center(-2x) + y_center(-2y) + stuff = x^2 + y^2, where the stuff is still nasty and nonlinear in terms of x_center, y_center, and R, but we don't need to care about it.  So we can just solve [-2x -2y 1][x_center, y_center, stuff]^T = [x^2 + y^2].

We could then back out R if we really wanted, but that doesn't help us much here.  Thankfully, the lstsq function can give us the residuals, which satisfies most of the conditions.  Subtracting the center and scaling by (R^2)^2 = R^4 ~ x^4 gives us translational and scale invariance.


This is positive definite because the squared residuals are nonnegative, and we're dividing by a square.  It tends toward 0 for circles and lines because we're fitting a circle.
I'm fairly sure it's not surjective, but I can't get a good bound. If there is an upper bound, we can map [0, bound) to the nonnegative reals (for example, with 1 / (bound - answer) - 1 / bound) for a few more bytes.
We subtract out the center, so it's translationally invariant.
We divide by x**4, which removes the scale dependence.
It's composed of continuous functions, so it's continuous.