Custom sorting function implemented as a jQuery plugin
Your code is nice and easy to understand. I'm going to break my review down into two sections: in the first section, I will suggest a different implementation, and in the second section, I will answer your questions.



New solution

In case you were not aware, JavaScript has a built-in function for sorting lists: Array.prototype.sort.

This function, basically, is used to take a list of data and sort it based on either it's built-in function, or a simple function you provide. This function takes two items from the array and returns either 1, 0, or -1 to tell sort how to place them in the array (the original array is mutated).

So, that being said, a much simpler way to sort these elements would be to do this:

// Sorts the items from greatest to least priority
$items.sort(function(a, b) {
    var aPriority = a.data("priority");
    var bPriority = b.data("priority");

    if(aPriority &gt; bPriority) {
        // Note this is -1 despite aPriority being greater; -1 moves aPriority earlier in the list, sorting it greatest to least
        return -1;
    }
    if(aPriority &lt; bPriority) {
        return 1;
    }
    return 0;
}


Not only is this simpler (and probably faster), it is also more idiomatic and easier to understand what is being done here.

Now that the native sort method is being used, there is no need for recursive placement. Just use a loop:

var positionElements = function($items, $container) {
    if (0 === $items.length) {
        return;
    }

    // ... $items sorting ...

    while($items.length &gt; 0) {
        $container.append($items[0]); // sorted greatest to least
        $items = $items.not($items[0]);
    }
};


This is much simpler now, and is a bit faster now that there is no recursion because there is only two places where the call stack is touched, rather than two places for every item in the list.



Questions


  Is my algorithm is optimal in terms of performance, or is it just terrible?


There are a few places where this code is not quite optimal. Look at this flowchart of your code:


Iterate through entire list to find highest element
Place that element and remove it from the list.
Go to 1 until list is empty.


It's not very efficient to be iterating through the entire list every time as long as there are still elements inside it. If, for example, there were 10 items in the list, the first you would iterate through 10 items, then 9 items, then 8 item... until you've iterated through 55 elements.

And that's why sorting the array first before working with it will be much faster (in fact, the highest voted question on Stack Overflow is about why processing a sorted array is faster).




  I'd like to create some automated tests for this feature. Is this code
  testable? What would be the best way/technology to use in order to
  test it?


There are many unit testing frameworks out there. Honestly, I'm not that familiar with jQuery, so my suggestion may not actually be your best bet (although, the jQuery website does make a reference to a unit testing framework: QUnit).

A framework I would recommend is Cheerio. My main reason for this is because it allows to easily load HTML into a virtual DOM and perform any normal DOM operations on it. This means you could load some test components, run your sorting code on them, and use Cheerio to read the DOM and make sure they are properly sorted.