The latter part of the name - felli - is Faroese for a small mountain, related to the English word 'fell' as in fell walking. 'Vørða' is a phrase used by Faroese regarding birdwatching. A 'Vørða' consists of five puffins, and since Vørðufelli is the fifth mountain on the north side of Sørvágsfirði, that explains how the mountain got its name.