GLSL function used to differentiate between pixels above/below a line I've recently acquired a program which is capable of using a GLSL fragment shader as a plugin. After a short amount of experimentation, I've realized that - for my use-case - I desperately need a function that does the equivalent of the following:


Takes two 2D points and calculates a line between them
Returns 1.0 if the current pixel is above (or below, callers choice) the line, and 0.0 otherwise. If the points are horizontally coincident, "above" == "to the left"


For this reason I've created the slice function. For my use-case this function has the potential to be called many, many times within a single shader, and should be As Performant As Possible™, but I am no GLSL guru...

My current, working code is below. The function arguments are as follows:


aspect: the current pixel's coordinate
p1, p2: the points defining the line
top: whether above or below the line returns 1.0




//Note: !(x ^^ y) == XNOR(x, y)
float slice(vec4 aspect, vec2 p1, vec2 p2, bool top) {
    // degenerate case - shade left or right
    if (p2.x == p1.x) return float(!(top ^^ aspect.x &lt; p1.x));

    // we must center the line over the leftmost point
    vec2 base = p2.x &gt; p1.x ? p1 : p2;

    //calculate the line's slope
    float slope = (p2.y-p1.y) / (p2.x-p1.x);

    //return 1.0 or 0.0 according the rules defined above
    return float(!(top ^^ (aspect.y - base.y)&gt;slope*(aspect.x - base.x)));
}


Can this code be made to run faster using only GLSL?
I would try using the dot product for your test. If you take the vector from p1 to p2 and form the dot product with the vector from p1 to aspect, it will return the cosine of the angle between them. The cosine of an angle is positive when the angle is between -90° and 90° and it's negative when the angle is in the other 180° of the circle. What this means is that when aspect falls on one side of the line it will be positive and on the other it will be negative. So your function could be rewritten as something like this:

float slice(vec4 aspect, vec2 p1, vec2 p2, bool top) {
    aspect /= aspect.w;    // &lt;- may be unnecessary if w is guaranteed to be 1.0
    vec2 v1 = aspect.xy - p1;
    vec2 v2 = p2 - p1;
    float cosTheta = dot(v1, v2);
    float result step(0.0, cosTheta);
    return top ? result : 1.0 - result;
}


I haven't tested this, but it removes all conditionals and logical operators. If you don't need the divide by w then there are no divisions, either. I think this could be faster, but you'll need to profile to be sure.

Note that this assumes that "on the line" can be taken as being on one side or the other. If you need to differentiate that case, I think you'll need either a conditional or some clever non-conditional code to extract that case.