Does the A* algorithm visit every node? I have been taught that the A* algorithm visits every node on the graph, like Dijkstra's does, prioritising nodes with the smallest cost. However, visualisations of the algorithm, such as this one, appear to show the algorithm stopping after finding the first path as the white blocks spreading out on the linked video do not cover the whole area before the path is highlighted, implying that not all nodes are visited.

Does the algorithm, therefore, not visit every node and instead stop after the first path is found? If so, is it not possible that a more optimum path could exist than the one found?
It only flood-fills the graph if there is no path to the end-node.

What would be the point of including a heuristic if you were just going to visit every node anyways?