Problems keyframing a simple cube, End keyframe cancels the starting keyframe
This kind of animation is - AFAIK - only possible with shape keys.
And as you can see - you can only add a keyframe to location (not to vertices or frame) and just in object mode. So the animation frame does relate to the whole object and it's location. Not to vertices or faces or edges.