Given a grouping of selected vertices in a mesh, how do I test if they form a contiguous edge loop?
I'd implement the following algorithm:


Give each vertex a unique ID.
Choose one of your grouping's vertices, and add its ID to a set() S.
Check all vertices ID that are connected by an edge to (2). Add these IDs to S.
Are any of the IDs in (3) the same as the ID in (2)? If yes, Return True.
If not, for each in (3), recursively do steps 2-5. If you run out of edges/vertices, return False.


Note: you use the vertices IDs in S to avoid choosing already processed vertices in step 5.

EDIT:

Pseudo-code could be something like:

grouping = [ selected vertices in a mesh ]

S = set()

for v in grouping.vertices():
    S.add(v.id)
    for vv in v.edges().vertex():
        if vv.id in S:
            return True
        else:
            S.add(vv.id)
return False