Remove vertex while using knife tool?
The knife tool has no undo support.
While it is possible to have undo history in a modal operator tool, at this time the knife tool does not. Nor is adding undo functionality planned.

However after confirming the knife tool cut, there are other modeling tools that can help you get around a misplaced vertex.


X > Dissolve Vertices, that will delete the selected vertex, or vertices and fill the surrounding geometry.
V if you accidentally snapped the vertex to a vertex already on the model, then you can rip the vertex and for a new one.
AltM > At Last or At First Merging, if you placed a vertex but wanted it to connect to a vertex already on the model, you can select the two vertices and merge them at the location of the "Good" vertex.
GG or ShiftV vertex slide, will let you slide the selected vertex along an edge.


Another tool that adds edges is Vertex Connect, it works by adding edges in a straight line between selected vertices.