What is a "closed set" in context of AI? How is it different from an open set or a frontier?
There are two variants of the A* algorithm:


The standard algorithm maintains two sets of vertices.  One called the "open set", and the other is called the "closed set".
An alternative version maintains just an open set, and doesn't use a closed set.  This requires that the heuristic function satisfy some some additional conditions.


See any standard treatment of A* for explanation of how it works and what the closed set is; e.g., A* search.