Net direction of dipole moment in oxygen difluoride
In H2O, as O is more electronegative than hydrogen, the resultant bond dipole is towards O, which means both the lone pair and bond pair dipole are acting in the same direction and the dipole moment of H2O is high. (1.84 D)
On the other hand,

In the case of F2O, the bond dipole is acting towards fluorine, so in F2O the lone pair and bond pair dipole are acting in opposition resulting in a low dipole (0.3 D)
So, basically, the dipole moment is towards fluorine atoms (which are more electronegative than oxygen atom). The reason why H2O has a higher dipole moment is that the difference between the electronegativity of O and H is higher than that of O and F, so water has a higher dipole moment even tho both water and oxygen difluoride have a similar bent molecular geometry.