Today however , the word mistress is used primarily to refer to the female companion of a man who is married to another woman ; in the case of an unmarried man it is usual to speak of a `` girlfriend '' . In modern times , however , the word mistress is used primarily to refer to the female lover of a man who is married to another woman ; in the case of an unmarried man it is usual to speak of a `` girlfriend '' or `` partner . ''