In which anime or manga was there a man who admired a fast killing sword technique but was mistaken
What you saw was an episode of Rurouni Kenshin

Specifically, 63: "The Legend of the Fireflies"



The episode is a perfect match to your description.

In that episode, Kenshin, the protagonist, encountered an old man who told him the exact story that you laid out in your post.

The man was already a hobbyist in swordsmanship, and participated in competitions with a wooden sword.  

He saw a robber who killed some people with a technique as "fast as lightning, never eve realizing that it was over for them"

He killed a person, and left his love interest behind because she disapproved of the killing. 

He killed more people, and eventually cut down the robber with ease.

When the robber died, the man saw the exact same expression on the robber's face, and he realized that he had surpassed him long ago and that the technique was just imagined, and not even real.   

When he got back to his home town, he learned that his love interest was dead.