How do I fill in the holes in wooden window frames?
If the window frames are made of wood, I would use wood filler to fill the holes.

You won't be able to conceal the holes from a forensic crime-scene examiner but that is not your goal. You need to restore the property to a state that will satisfy a reasonable owner that you have looked after it and kept it in good repair so that it can be rented out once you have terminated your lease.

It is always wise, before drilling holes in someone else's property, to ask the owner.

If in doubt, ask the owner what she would like you to do.